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Abstract 


This  study,  with  support  of  the  Aeronautical  Systems  Center,  builds  a  scientific  decision 
support  system  intended  for  Air  Combat  Command.  Past  decision-making  techniques  are 
discussed  and  their  limitations  are  explained.  Two  multiattribute  decision-making  techniques, 
analytical  hierarch  process  and  value  focused  thinking,  are  studied.  Due  to  limitations  of  analytical 
hierarchy  process,  value  focused  thinking  is  accepted  as  the  best  fit  for  our  problem.  A  value 
hierarchy  model  is  developed  using  value  focused  thinking  for  selecting  the  most  beneficiary  F-16 
engine  modifications  depending  on  anticipated  Air  Combat  Command  values.  Ten  modifications 
are  ranked  using  value  model  to  validate  the  process.  Cost  is  involved  in  the  rankings  to  show  the 
benefit  per  dollar  invested.  Optimization  techniques  are  used  to  form  various  effective 
modification  sets  due  to  changing  budget  constraints.  Sensitivity  analyses  show  that  the  model  is 
weight  sensitive.  This  study  proves  that  multiattribute  decision-making  techniques  and  particularly 
value  focused  thinking  approach  can  be  used  to  create  a  scientific  decision  support  system  for  Air 
Combat  Command. 
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TECHNOLOGY  SELECTION  FOR  ENHANCING  F-16 
CAPABILITY:  AN  ANALYSIS  USING  VALUE  FOCUSED 

THINKING 


Chapter  1 :  Introduction 


1.1  Background 

In  today’s  technology-driven  world,  organizations  that  perform  research  and 
development  (R&D)  are  forced  to  make  tough  and  risky  decisions  about  their  financial 
investments.  Decision-makers  may  face  unfamiliar  technologies  when  making  decisions. 
If  the  decision-maker  does  not  understand  the  technology,  the  company  may  face  budget 
cuts  and  lose  market  share  to  the  competition.  Decision-makers  need  to  show  that  their 
decisions  have  an  additive  value  to  the  organization’s  existing  position.  Bad  decisions 
might  result  in  losing  money  for  short  term  and  can  lead  to  losing  capabilities  of  the 
company  in  the  long  term.  The  goal  is  to  choose  technologies  to  benefit  their  competitive 
advantage. 

The  military  organizations  throughout  the  world  are  spending  a  vast  amount  of 
money  for  R&D  projects.  Military  decision-makers  must  understand  the  value  for  a 
particular  R&D  project  since  poor  R&D  project  selection  can  lead  to  losses  of  lives  and 
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losses  of  strategic  objectives.  Modernization  of  equipment  is  a  good  example  of  R&D 
project  selection  in  military.  New  technologies  have  to  be  integrated  into  existing 
equipment  to  keep  their  benefit  for  the  armed  forces.  However,  selecting  which 
technologies  is  not  an  easy  process.  Multiple  objectives  must  be  considered  (costs, 
safety,  etc.).  This  research  builds  a  mathematical  model  to  support  the  decision  process 
of  selecting  F-16  engine  modifications. 

The  goal  of  decision  analysis  is  to  help  individuals  make  good  decisions 
(Ragsdale,  2001:  714).  Unforeseeable  circumstances  and  complex  systems  obscure 
decisions  and  may  result  in  poor  outcomes.  The  better  the  decision-maker  understands 
the  system,  the  better  the  decision  will  be  for  selecting  technologies.  The  decision¬ 
making  process  requires  decomposing  the  problem  and  creating  a  framework  for 
achieving  the  objectives  of  the  company  or  organization. 

The  scientific  method  provides  this  framework.  Often,  individuals  rely  on 
intuitive  methods  to  solve  the  problems  of  daily  life  by  comparing  between  alternatives. 
However,  R&D  selection  projects  include  multiple  competing  objectives  and  tradeoffs 
are  required  between  these  objectives.  More  than  an  intuitive  method,  we  need  a 
philosophy,  a  systematic  way  to  approach  decision-making  process.  A  key  to  good 
decision-making  is  to  provide  a  structured  method  for  incorporating  the  information, 
opinions,  and  preferences  of  the  various  relevant  people  into  the  decision-making  process 
(Kirkwood,  1997:1). 

Technology  selection  models  help  the  decision-maker  choose  between  evolving 
technologies.  Some  mathematical  programming  approaches  have  been  used  for 
technology  selection  models  in  the  past  (Weingartner,  1963;  Lorie  &  Savage,  1955). 
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Linear  programming  is  a  good  example  of  a  mathematical  approach,  which  is  used  for 
portfolio  selection.  Such  approaches,  however,  do  not  capture  the  different  aspects  of  the 
competing  alternatives  (Baker  and  Freeland,  1975).  New  approaches  have  been 
developed  to  assist  in  decision-making  process  over  the  last  decades. 

New  analysis  methods  concentrate  on  the  decision-making  process  for  multiple 
objectives,  requiring  tradeoffs  among  competing  objectives.  These  methods  can  be 
applied  to  decision-making  processes  for  new  technologies.  This  research  examines  and 
implements  strategic  decision-making  tools  for  multiple  objectives. 

1 .2  Problem  Statement 

The  purpose  of  this  thesis  is  to  develop  a  mathematical  model  to  pick  a  set  of  best 
modifications  for  F-16  capability  enhancement.  Values  (what  is  important  to  decision¬ 
makers  about  choosing  the  engine  modifications)  are  used  in  the  model  to  quantify  how 
well  various  modifications  enhance  F-16  capability.  The  model  will  provide  insights  to 
the  F-16  System  Program  Office  (SPO)  in  selecting  sets  of  modifications  for  future 
integration  into  the  aircraft.  The  main  considerations  for  the  modifications  are  combat 
capability,  safety,  and  operational  costs. 

1 .3  Objective  and  Scope  of  the  Research 

The  main  objective  is  to  use  value  focused  thinking  (VFT)  to  develop  an  overall 
modification  selection  model  with  the  SPO  and  Aeronautical  Systems  Center  (ASC). 

The  model  reflects  the  anticipated  needs  of  Air  Combat  Command  (ACC).  The  goal  of 
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the  model  is  to  rank  modification  alternatives  based  on  combat  capability,  safety 
enhancements  and  operations  costs. 

1 .4  Summary  and  Organization 

Chapter  2  documents  literature  reviewed  for  the  purpose  of  this  study.  It  explains 
the  F-16  safety  process,  F-16  combat  capability  highlights,  and  cost  aspects  for  the 
model.  Decision  analysis  and  R&D  selection  models  are  also  introduced  with  some 
selected  applications.  An  in-depth  discussion  is  provided  on  the  value  focused  thinking 
approach.  Finally,  resource  allocation  models  are  highlighted. 

Chapter  3  applies  the  results  of  Chapter  2  to  our  specific  problem.  First,  how  to 
choose  an  appropriate  model  for  R&D  portfolio  selection  is  discussed.  Next,  a  detailed 
discussion  is  provided  on  developing  a  value  hierarchy  model  for  the  F-16  capability 
enhancement  model.  In  the  process,  in-depth  insights  on  VFT  and  its  procedures  are 
explained.  Single  dimensional  value  functions  are  formed  depending  on  the  preferences 
of  the  decision-maker.  Weights  of  the  model  are  introduced  using  examples. 
Assumptions  of  the  model  are  explained  to  complete  the  chapter. 

Chapter  4  shows  the  results  of  an  illustrative  analysis  on  a  small  subset  of 
modifications  by  choosing  the  best  portfolio.  Acquisition  cost  for  an  engine  and  total 
cost  for  the  modifications  are  studied  in  a  benefit/cost  ratio  analysis  to  provide  better 
insights  about  the  value  of  the  modifications.  Integer  programming  is  used  for  selecting 
effective  modification  sets  using  budget  constraint. 
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Chapter  5  concludes  the  study  and  provides  final  insights,  recommendations  and 
areas  for  future  research.  Specifically  we  focus  on  engine  modifications.  This  chapter 
also  includes  a  lessons  learned  or  process  tips  section  to  help  future  researchers.  The 
appendices  provide  more  detailed  information  on  the  data  gathering  and  model  building 
process. 
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Chapter  2:  Literature  Review 

Nothing  is  more  difficult ,  and  therefore  more  precious,  than  to  be  able  to  decide. 


NAPOLEON, 
Maxims,  1804 


2.1  Introduction 

This  chapter  discusses  the  F-16  program  and  the  decision-making  concepts  used 
in  this  study.  Safety,  combat  capability  and  operations  costs  are  explained  to  introduce 
the  F-16  model  terminology.  Decision  analysis  is  discussed  and  an  in-depth  discussion  is 
provided  on  the  VFT  approach  to  decision-making.  Analytical  hierarchy  process  (AHP) 
is  also  introduced  as  an  alternative  decision  making  process.  Some  examples  from  the 
literature  that  discuss  these  methods  as  applied  to  decision  analysis  in  recent  history  are 
presented.  Characteristics  and  limitations  of  resource  allocation  and  R&D  models  are 
examined  to  complete  the  chapter. 

2.2  F-16  Safety  Program 

The  F-16  system  safety  program  plan  (SSPP)  is  a  working  document  that  reflects 
the  policies  and  procedures  that  identify  and  eliminate  or  control  identified  hazards  for 
the  common  configuration  implementation  program  (CCIP)  and  other  F-16  acquisition 
reform  programs.  The  SSPP  is  structured  in  accordance  with  a  military  safety  document, 
MIL-STD-882C,  to  provide  guidance  for  accomplishing  the  F-16  safety  program  by  the 
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assigned  system  safety  engineers.  The  safety  team  monitors  the  performance  and 
progress  of  the  program  and  modification  changes. 

2.2.1  Hazard  Control  Decision  Process 

Figure  2.1  illustrates  the  hazard  control  decision  process.  The  key  point  to  note 
from  this  decision  process  flow  chart  is  that  the  primary  (and  majority  of)  decisions  are 
normally  made  informally  at  the  working  level  between  system  safety  engineers  and 
design  engineers.  When  agreement  cannot  be  reached  at  the  working  level,  system  safety 
issues  are  formally  documented  and  elevated  for  a  management  decision.  The  hazard 
control  decision  process  is  terminated  when: 

•  The  hazard  is  eliminated  or  adequately  controlled  by  changing  the  design  ,  or 

•  The  risk  is  accepted,  documented  and  reported  to  the  F-16  SPO  and  Ogden  safety 
offices. 
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Figure  2.1  Hazard  Control  Decision  Process  ( Lockheed  Martin 
Tactical  Aircraft  Systems  Report) 


2.2.2  Risk  Assessment 


Hazard  severity  categories,  defined  in  Table  2.1,  and  hazard  probability  levels, 
defined  in  Table  2.2,  are  in  accordance  with  MIL-STD-882C,  paragraphs  4.5.1  and  4.5.2. 
Lockheed  Martin  Tactical  Aircraft  Systems  (LMTAS)  developed  a  quantitative  hazard 
rate  for  each  probability  level,  as  listed  in  Table  2.1.  The  quantitative  hazard  rates  are 
based  on  total  USAF  F-16  fleet  flight  hours  expected  to  be  flown  at  completion  of  the  F- 
16  program.  Flight  fleet  hours  represent  the  total  hours  flown  by  all  USAF  F-16s  in 
the  fleet.  For  applications  that  are  not  applicable  to  the  entire  fleet  (e.g.,  equipment  only 
on  certain  blocks  of  aircraft),  the  proportional  number  of  flight  hours  can  be  used  to 
derive  the  appropriate  hazard  rates. 


Table  2.1  Hazard  Severity  Categories 


Description 

Category 

Definition 

Catastrophic 

1 

May  cause  death,  system  loss,  or  severe 
environmental  damage. 

Critical 

2 

May  cause  severe  injury,  severe  occupational 
illness,  or  major  system  or  environmental  damage. 

Marginal 

3 

May  cause  minor  injury,  minor  occupational 
illness,  or  minor  system  or  environmental  damage. 

Negligible 

4 

Will  not  cause  injury,  occupational  illness,  or 
system  or  environmental  damage. 

Any  time  a  hazard  is  identified  that  is  judged  to  be  inadequately  controlled,  the 
procedure  in  Figure  2.1  is  followed. 
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Table  2.2  Hazard  Probability  Categories  &  Rates 


Description 

Level 

Fleet  or  Inventory 

Rate*(Per  Flight  Hour) 

Frequent 

A 

Continuously  Experienced 
(Greater  than  500  Occurrences) 

5.  0  E-05  to  °° 

Probable 

B 

Will  Occur  Frequently 
(Between  5-500  Occurrences) 

5.  0  E-07  to  5.  0  E-05 

Occasional 

C 

Will  Occur  Several  Times 
(Between  1-5  Occurrences) 

1.  0  E-07  to  5.  0  E-07 

Remote 

D 

Unlikely  But  Can  Reasonably  Be 
(Between  .  01-1  Occurrences) 

I.  0  E-09  to  1.  0  E-07 

Improbable 

E 

Unlikely  To  Occur,  But  Possible 
(Less  than  .  01  Occurrences) 

Less  than  1.  0  E-09 

*Rate  based  upon  10,000,000 flight  hours  for  USAF  F-16  fleet  life.  Adjust  rate  if  application  is  less  than 
fleet  life. 


A  risk  assessment  combines  the  hazard  severity  and  hazard  probability.  The 
hazards  in  Table  2.3  (hazard  risk  index  of  1A,  IB,  1C,  2A,  2B,  or  3A),  have  unacceptable 
risk  and,  therefore,  require  additional  hazard  controls  to  reduce  their  risk.  The  light 
shaded  hazards  in  Table  2.3  (hazard  risk  index  of  ID,  2C,  3B,  or  3C)  have  undesirable 
risk  and,  therefore,  require  implementation  of  the  hazard  control  decision  process  in 
Figure  2.1.  The  unshaded  hazards  in  Table  2.3  (hazard  risk  index  of  IE,  2E,  3D,  3E,  4A, 
4B,  4C,  4D,  or  4E)  have  acceptable  risk  and  do  not  require  further  action. 
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Table  2.3  Hazard  Risk  Assessment  Matrix  with  acceptability  criteria 


Catastrophic 

1 

Critical 

2 

Marginal 

3 

Negligible 

4 

A  Frequent 

|l 

IA**H 

4A 

B  Probable 

| 

lb**H 

4B 

C  Occasional 

Hi 

lc**H 

^■2C*Hi 

gjg 

4C 

CIS 

|2D*| 

3D 

4D 

IE 

2E 

3E 

4E 

2.3  Combat  Capability 

There  are  many  documents  about  combat  capability  in  the  literature.  The  USAF 
web  page  is  the  most  important  resource  for  this  study.  Combat  capability  is  the  realized 
capability  of  a  force  at  any  instant  of  time  to  achieve  combat  results  in  furtherance  of  a 
specific  mission  against  a  specific  enemy  force  in  a  specific  combat  environment. 

Combat  capability  is  accepted  as  the  actual  instantaneous  force  that  influences  the  combat 
situation  (The  Nation’s  Air  Force,  2000). 

The  Air  Force  usually  provides  the  quickest  response  and  longest-range  forces 
available  to  the  armed  forces.  The  USAF  can  deter,  deploy  for  influence,  or  employ  with 
lethal  force  anywhere  in  the  combat  region.  Achieving  air  superiority  and  conducting 
precision  attacks  are  key  elements  in  fighting  and  winning  the  war.  Air  superiority 
includes  the  ability  to  protect  our  forces  against  any  kind  of  attacks,  such  as  ballistic 
missile  attacks.  Precision  attack  is  the  combination  of  precise  target  acquisition, 
munitions,  and  weapons  delivery.  Increasing  combat  capability  is  the  main  goal  for  the 
Air  Force  acquisition  process.  The  USAF  has  to  pursue  modifications  that  will  increase 
F-16  combat  capability  (The  Nation’s  Air  Force,  2000). 
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2.4  Operations  Cost 


Cost  is  a  major  concern  and  a  driving  factor  in  the  development  and  operation  of  a 
weapon  system.  Also  of  concern  are  the  requirements  of  the  system  being  planned  and 
their  relation  to  overall  cost.  Developing  the  connection  between  requirements  and  cost 
quantification  is  not  a  trivial  problem.  The  operational  costs  are  those  that  happen  during 
the  project-life.  They  are  not  depreciable  costs  and  are  used  to  maintain  the  whole 
process  in  operation.  The  main  items  of  operations  cost  are  personnel,  consumables 
(parts),  and  maintenance  (Costing  and  charging  for  research,  1995). 

The  decision  to  field  a  new  system  requires  a  commitment  to  support  that  system 
for  years  into  the  future.  Decisions  to  develop,  procure,  and  support  new  systems  are 
based  on  many  factors,  one  of  which  is  the  projected  cost  of  the  systems  over  their 
operational  lifetime.  Operating  and  support  costs  normally  constitute  a  major  portion  of 
system  life-cycle  costs  and,  therefore,  are  critical  to  the  evaluation  of  acquisition 
alternatives. 

2.5  Decision  Analysis 

Selecting  the  best  modifications  to  enhance  the  F-16  capabilities  is  a  complex 
decision  problem.  Safety,  combat  capability,  and  operations  costs  are  different  aspects  of 
this  decision.  Thinking  about  the  tradeoffs  easily  shows  that  one  cannot  simply  rely  on 
instinct  to  decide.  The  obvious  reason  for  studying  decision  analysis  is  that  carefully 
applying  its  techniques  can  lead  to  better  decisions  (Clemen,  1996:  3). 
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In  general,  decision-making  involves  the  following  concerns: 

•  Planning 

•  Generating  a  set  of  alternatives 

•  Setting  priorities 

•  Choosing  a  best  policy  after  finding  a  set  of  alternatives 

•  Allocating  resources 

•  Determining  requirements 

•  Predicting  outcomes 

•  Designing  systems 

•  Measuring  performance 

•  Insuring  the  stability  of  a  system 

•  Optimizing 

•  Resolving  the  conflict 


(Saaty,  1990:5) 

Each  individual  decision  has  its  own  defining  frame.  The  decision  analysis 
process  helps  the  decision-maker  pick  the  best  alternatives  within  the  decision  frame.  A 
decision  is  considered  difficult  due  to: 

•  Complexity 

•  Uncertainty  in  the  situation 

•  Multiple  objectives 

•  Different  perspectives  and  different  conclusions. 


(Clemen,  1996:3) 

Decision  analysis  provides  “  structure  and  guidance  for  systematic  thinking  in 
difficult  situations”  (Clemen,  1996:4).  Decision  analysis  is  not  designed  to  make  the 
decision  for  the  decision-maker.  “The  basic  presumption  of  decision  analysis  is  not  all  to 
replace  the  decision-maker’s  intuition,  to  relieve  him  or  her  of  the  obligations  in  facing 
the  problem,  or  to  be,  worst  of  all  competitor  to  the  decision-maker’s  personal  style  of 
analysis,  but  to  complement,  augment  and  generally  work  alongside  the  decision-maker 
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in  exemplifying  the  nature  of  the  problem.  Ultimately,  it  is  of  most  value  if  the  decision¬ 
maker  has  actually  learned  something  about  the  problem  and  his  or  her  own  decision¬ 
making  attitude  through  the  exercise”  (Bunn,  1984:8). 

The  process  of  decision-making  is  complicated  when  dealing  with  multiple 
objectives.  The  problems  are  complex,  and  humans  commonly  rely  on  intuition  to  solve 
problems.  Intuition  can  fail  when  the  decision-maker  must  make  tradeoffs  between  the 
competing  alternatives.  Therefore,  we  need  a  well-organized  way  to  make  better 
decisions. 

An  effective  decision-making  process  will  fulfill  these  six  criteria: 

•  It  focuses  on  what  is  important. 

•  It  is  logical  and  consistent. 

•  It  acknowledges  both  subjective  and  objective  factors  and  blends  analytical 
with  intuitive  thinking. 

•  It  requires  only  as  much  information  and  analysis  as  is  necessary  to  resolve  a 
particular  dilemma. 

•  It  encourages  and  guides  the  gathering  of  relevant  information  and  informed 
opinion. 

•  It  is  straightforward,  reliable,  easy  to  use  and  flexible. 


(Hammond,  Keeney,  Raiffa  ,  1999:4) 
Figure  2.2  explains  the  basic  steps  of  the  decision-making  process. 

The  techniques  for  decision  analysis  can  help  you  make  better  decisions.  However,  one 
should  always  remember  that  they  do  not  guarantee  that  good  outcomes  will  always 
occur  as  a  result  of  those  decisions.  Using  a  structured  approach  in  making  decisions 
enhances  our  intuition  about  the  decision  problems  we  face.  As  a  result,  it  is  reasonable 
to  expect  better  outcomes  to  occur  more  frequently  when  using  a  structured  approach  to 
decision-making  than  if  we  make  decisions  in  a  haphazard  manner  (Ragsdale,  2001:714). 
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Corrective  Action  and  Revision  of  Norms 
(Applied  to  New  Decisions) 


Figure  2.2  The  Decision  Process  (Nutt  ,1989  :  408) 
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2.6  R&D  Selection  Models 


A  model  is  needed  to  support  technology  selection  for  the  F-16  system  program. 
Bretschneider  (1993)  defines  two  types  of  analysis  in  R&D  assessments.  He  defines  ex 
ante  R&D  as  those  evaluations  occurring  before  any  R&D  activity  and  ex  post  R&D  as 
those  evaluations  occurring  a  after  a  project  has  been  completed.  Ex  ante  analysis 
focuses  on  outcomes  and  impacts  and  is  used  to  select  among  competing  projects.  Based 
on  these  definitions,  this  study  is  going  to  be  an  ex  ante  analysis. 

Bretschneider  further  divide  ex  ante  studies  into  valuation,  or  benefit 
measurement  models  and  resource  allocation  models  (Bretschneider  ,  1993:124). 
Valuation  models  are: 

•  Models  that  develop  a  measure  of  value  thorough  a  comparative  technique 

•  Models  that  are  based  on  obtaining  a  multidimensional  score 

•  Techniques  that  link  a  project’s  value  to  the  overall  economic  objective  of  the 
firm  or  organization 

Resource  Allocation  models  are: 

•  Constrained  optimization  models 

•  Emulations  of  organizational  and  human  processes  (simulation) 

•  Ad-hoc  in  nature 


VFT  and  AHP  are  two  types  of  multidimensional  scoring  models.  VFT  places 
values  in  an  hierarchical  structure  and  quantifies  them  with  evaluation  measures  to  create 
a  value  model.  Alternatives  are  scored  using  the  value  model  allowing  quantification 
based  on  achievement  of  values.  AHP,  on  the  other  hand,  structures  priorities.  Pairwise 
comparisons  are  used  to  rank  the  alternatives. 

A  weakness  of  VFT  is  the  inability  to  systematically  check  for  consistency  of 
judgments  (Belton,  1986:18).  On  the  other  hand,  AHP  is  known  to  have  theoretical 
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problems.  Belton  compares  VFT  with  AHP  and  finds  AHP  has  a  major  weakness  in  the 
manner  of  asking  questions  to  determine  criteria  weights  and  in  the  assumption  that  one 
can  use  a  ratio  to  compare  measurement  scores. 

Bard  (1992)  writes  that  AHP  is  simpler  to  use  for  an  inexperienced  decision¬ 
maker  compared  to  VFT.  However,  AHP  always  results  in  additive  weighted  value 
functions  and  should  not  be  used  for  risky  decisions,  where  as  VFT  is  not  restricted  in 
these  ways  (Belton,  1986:10),  (von  Winterfieldt  and  Edwards,  1986:  275-276).  VFT  can 
also  create  new  alternatives  that  improve  the  decision  context.  The  ability  to  create  new 
alternatives  prevents  the  decision  context  from  being  “anchored”  to  narrowly  defined 
alternatives.  A  VFT  value  model  can  be  used  systematically  to  probe  new  alternatives 
that  may  be  better  than  those  first  identified  without  systematic  analysis  (Keeney, 
1994:38-39). 

Cost  benefit  analysis,  rate  of  return  analysis,  and  risk  assessment  are  the  other 
traditional  types  of  R&D  selection  models  that  are  in  the  literature. 

2.6.1  Analytical  Hierarchy  Process 

Value  function  assessments  of  the  model  and  ranking  of  the  alternatives  are  not 
really  distinctive  with  the  AHP  approach.  The  decision-maker  decides  which  alternative 
is  better,  A  or  B,  within  a  specific  evaluation  consideration.  The  decision-maker  uses  a 
nine-point  scale  to  do  this  comparison.  This  scale  shows  the  performance  of  one 
alternative  with  respect  to  another.  A  mathematical  process  is  used  to  select  the  best 
alternative  among  the  various  pairwise  comparisons. 


17 


The  AHP  approach  uses  pairwise  comparisons.  This  is  a  powerful  approach  since 
the  human  mind  excels  at  making  comparisons  between  two  alternatives.  Our  focus  is  on 
decisions  where  there  are  two  or  more  objectives  in  competition  and  many  times  those 
competing  objectives  require  tradeoffs.  However,  the  time  for  the  decision  processes  to 
produce  the  most  effective  result  is  equally  important. 

The  AHP  process  can  require  a  vast  amount  of  time  depending  on  the  number  of 
alternatives  and/or  evaluation  measures.  Decision  models  with  many  alternatives  and 
evaluation  measures  need  a  lot  of  time  to  make  pairwise  comparisons.  Additionally, 
decision  makers  are  often  inconsistent  when  making  pairwise  comparisons  between 
competing  alternatives. 

Another  objection  to  the  AHP  is  rank  reversal,  which  is  considered  the  most 
significant  flaw  in  the  AHP  process.  The  addition  of  a  new  alternative  can  change  the 
ranking  of  the  existing  alternatives,  even  though  the  evaluation  measures  stay  the  same. 

It  can  be  shown  that  rank  reversal  is  normal  mathematically.  However,  the  main  issue  is 
if  rank  reversal  has  a  big  impact  on  our  decision  or  not.  It  is  not  desirable  that  the  top 
ranking  alternatives  change  every  time  something  is  added  or  deleted  from  the  model. 

Another  shortcoming  of  AHP  is  the  use  of  approximation  methods  which  impact 
the  given  decision.  Therefore,  the  one  who  has  a  very  important  and  weight-  sensitive 
decision  to  make  should  be  careful  when  using  AHP,  since  the  precision  of  the  decision 
depends  on  some  approximation.  Overall,  these  model  deficiencies  force  a  decision¬ 
maker  to  look  for  another  technique. 
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2.6.2  Value  Focused  Thinking 


VFT  is  based  on  the  concept  of  values.  Values  measure  how  desirable  or 
undesirable  an  alternative  is,  based  on  the  consequences  the  alternative  brings  out.  Value 
focused  thinking  essentially  consists  of  two  activities:  first  deciding  what  you  want  and 
then  figuring  out  how  to  get  it  (Keeney,  1992:3-4). 

The  goal  is  to  incorporate  the  value  into  an  objective  technique  for  decision¬ 
making.  Once  this  is  accomplished,  we  can  compare  the  competing  alternatives. 

VFT  is  a  PrOACT  approach  to  make  smart  choices.  Hammond,  Keeney,  and 
Raiffa  use  PrOACT  as  an  abbreviation  word  for  Problem,  Objectives,  Alternatives, 
Consequences,  and  Tradeoffs  (Hammond,  Keeney,  Raiffa,  1999:  7-9). 

We  can  further  define  each  of  these  terms. 

•  Problem:  “What  must  you  decide?”  is  the  most  important  question  to  define  your 
problem.  The  framework  of  the  decision  depends  on  the  complexity  and  assumptions 
of  the  problem. 

•  Objectives:  It  is  asking  yourself  what  you  most  want  to  accomplish.  Your  values, 
interests,  and  concerns  will  clear  your  objectives. 

•  Alternatives:  They  are  different  courses  of  action  that  you  might  take.  However,  you 
have  to  understand  that  your  best  decision  can  be  no  better  than  your  best  alternative. 

•  Consequences:  Consequences  are  the  answer  to  the  question  of  “How  well  do  your 
alternative  satisfy  your  objectives?”.  Consequences  will  help  you  find  the  best 
alternative. 
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•  Tradeoffs:  A  balance  is  needed  in  every  decision  problem.  Setting  some  priorities 
between  the  competing  objectives  will  make  the  problem  easier.  Tradeoffs  set  this 
balance  due  to  your  priorities. 

VFT  helps  you  to  see  both  the  tangible  and  intangible  aspects  of  your  decision 
situation  more  clearly  and  translate  all  pertinent  facts,  feelings,  opinions,  beliefs,  and 
advice  into  the  best  possible  choice  (Hammond,  Keeney,  Raiffa,  1999:5). 


2. 6.2.1  Examples  of  VFT 

A  difficult  problem  for  National  Aeronautics  and  Space  Administration  (NASA) 
was  to  choose  the  future  space  mission.  There  were  numerous  stakeholders  involved  in 
addressing  multiple  uncertainties.  The  decision  required  some  tradeoffs  among  the 
objectives.  NASA  identified  and  prioritized  their  objectives  as  shown  in  Table  2.4. 


Table  2.4  NASA’s  Objectives  (Keeney  ,  1998) 


Objective 

Ranked 

Relative 

Enhance  National  Pride 

1 

100 

Aid  National  Defense 

9 

20 

Promote  International  Prestige 

8 

35 

Foster  International  Cooperation 

7 

40 

Create  Economic  Benefits 

5 

50 

Advance  Scientific  Knowledge 

4 

60 

Promote  Education 

6 

45 

Provide  Excitement  and  Drama 

2 

90 

Maintain  Fiscal  Responsibility 

3 

70 

The  objectives  were  scored  on  a  scale  between  0-100  and  ranked  depending  on 
their  relative  importance.  The  possible  alternatives,  four  different  missions,  were 
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compared  using  the  objectives.  The  missions  were  ranked  in  priority  order.  Tradeoffs 
among  alternatives  were  discussed.  Paired  comparisons  were  made  between  some 
competing  alternatives  using  experts’  opinions.  The  results  are  shown  in  Table  2.5. 


Table  2.5  Ranking  of  NASA  missions  in  terms  of  their  consequences  for  each 

NASA  objective 


Attribute 

A 

B 

C 

D 

National  Pride 

4 

3 

2 

1 

National  Defense 

2 

4 

1 

3 

International 

Prestige 

4 

3 

2 

1 

International 

Cooperation 

1 

2 

4 

3 

Economic  Benefits 

1 

4 

2 

3 

Scientific 

Knowledge 

2 

1 

3 

4 

Education  and 
Excellence 

3 

2 

4 

1 

4 

3 

2 

1 

1 

2 

3 

4 

VFT  has  been  used  in  many  other  decision  problems.  The  Department  of  Energy 
has  used  VFT  extensively.  Transporting  nuclear  wastes  and  examining  air  pollution  are 
some  other  applications  of  VFT  in  the  literature. 

Many  Department  of  Defense  (DoD)  agencies  have  used  VFT  approach  to  make 
decisions  in  the  past.  One  important  VFT  study  is  the  safety  of  landing  an  aircraft  which 
was  done  by  Yntema  and  Klem  (1965).  The  safety  of  landing  an  aircraft  depends  on 
many  factors:  wind,  visibility,  ceiling,  other  aircraft  in  the  vicinity,  and  so  on.  Ytnema 
and  Klem  attempted  to  quantify  the  safety  of  various  situations  that  differed  in  terms  of 
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ceiling,  visibility,  and  the  amount  of  fuel  that  would  remain  at  touch  down  on  a  normal 
landing. 

The  decision-makers  were  20  experienced  Air  Force  pilots.  The  utility  functions 
for  ceiling,  visibility  and,  fuel  were  assessed  depending  on  the  pilot’s  preferences.  Each 
pilot  was  presented  with  40  pairs  of  consequences  and  asked  to  pick  the  preferable  one  of 
each  pair.  The  responses  compared  with  the  utility  functions.  Yntema  and  Klem 
concluded  that  the  results  were  satisfactory  (Keeney  and  Raiffa  ,  1993:418). 

Major  Brian  Sperling  used  VFT  approach  to  build  a  model  for  consistently 
evaluating  Army  aviation  hazards  in  an  Air  Force  Institute  of  Technology  (AFIT)  thesis. 
The  model  integrated  the  Army’s  risk  management  process  and  the  Director  of  Army 
Safety’s  values  into  the  Army  Safety  Center’s  resource  allocation  process  to  reduce 
mishap  rates.  The  model  identified  the  most  severe  aviation  accidents  and  helped  Army 
Safety  Center  to  define  the  most  valuable  controls  to  reduce  these  accidents.  The  study 
validated  the  concept  of  using  value  focused  thinking  to  rank  accidents  and  hazards  while 
developing  a  cornerstone  for  research  efforts  with  the  Army  Safety  Center  for  a  proposed 
five  year  plan  (Sperling,  1999). 

2. 6.2.2  Value  Hierarchy  Development  Methods 

A  value  model  includes  qualitative  and  quantitative  relationships.  A  value  model 
should  be  developed  from  first  principles,  sound  logic,  reasoned  judgments,  and  carefully 
acquired,  consistent  data  (Keeney  ,  1992:130). 
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There  are  several  accepted  ways  to  develop  value  hierarchies.  Kirkwood 
identifies  a  top  down  or  a  bottom  up  approach  as  possibilities  (Kirkwood,  1997:19-23). 
The  top  down  approach  is  used  when  the  alternatives  are  not  well  specified.  Typically, 
information  for  this  method  comes  from  mission,  vision  and  strategic  documents.  The 
model  may  be  built  based  on  documented  information.  This  process  is  called  the  “Gold 
Standard”  when  used  to  develop  a  value  hierarchy  (Parnell  et.al.,  1998:1338) 

In  the  bottom  up  approach  discussed  by  Kirkwood,  alternatives  are  known  and 
can  be  examined  to  determine  how  they  differ  from  each  other.  Another  approach  similar 
to  Kirkwood’s  is  to  determine  what  task  the  organization  performs  with  a  group  of  people 
and  name  the  tasks  using  verbs.  This  approach  uses  the  preferences  of  the  experts  and 
decision-makers.  If  the  documentation  does  not  contain  enough  information  to  build  the 
model,  this  approach  may  be  used  effectively.  This  approach  is  called  the  “Silver 
Standard”  (Pamell  et.al.,  1998:  1340).  Figure  2.3  shows  a  value  hierarchy  model 
example.  Strategic  objective  is  divided  into  three  different  objectives.  Objectives  in  the 
model  are  made  up  of  different  aspects  of  the  decision  problem.  The  lowest  level  in  the 
value  hierarch  includes  different  metrics  used  for  quantifying  the  alternatives’ 
achievements  (measure  of  merits).  Other  terms  that  are  sometimes  used  for  metrics  are 
attribute,  evaluation  measure,  and  measure  of  effectiveness.  A  metric  is  used  to  measure 
the  degree  of  attainment  of  an  objective. 
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Figure  2.3  Value  Hierarchy  (Kloeber,  Parnell:  VFT  Brief) 


The  value  model  is  expected  to  have  various  properties  to  work  properly.  The 
desirable  properties  for  a  value  hierarchy  should  be  completeness,  nonredundancy, 
decomposability,  operability,  and  small  size  (Kirkwood,  1997:16).  Keeney’s 
explanations  for  the  desirable  properties  of  the  model  are  summarized  below 
Keeney(1992). 

•  Completeness:  There  are  two  different  requirements  for  a  model  to  be  complete:  (1) 
Each  tier  must  adequately  cover  all  concerns  necessary  to  evaluate  the  overall 
objective,  (2)  Lowest  tier  evaluation  considerations  adequately  measure  the  degree  of 
attainment  of  their  associated  objectives.  “A  set  of  objectives  is  complete  if  the 
knowledge  of  the  possible  consequences  with  respect  to  each  of  the  sub-objectives 
provides  a  description  of  all  the  implication  of  interest  when  an  alternative  is  selected 
in  a  decision  problem”  (Keeney,  1992:58). 
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•  Nonredundancy:  No  two  evaluation  considerations  in  the  same  tier  should  overlap.  A 
nonredundant  (mutually  exclusive)  hierarchy  means  that  no  data  in  the  evaluation  are 
double  counted  in  the  model.  This  can  be  difficult,  because  double  counting  can 
occur  in  two  ways.  One  is  double  counting  the  possible  impacts  of  the  alternatives 
and  the  other  is  double  counting  the  values  of  those  impacts.  Eliminating  any 
redundancies  reduces  the  number  of  objectives  and  reduces  the  effort  required  for 
data  gathering. 

•  Decomposability:  The  preference  of  one  evaluation  consideration  should  not  depend 
on  the  other  one.  Lack  of  decomposability  causes  difficulties  in  the  decision-making 
process,  especially  for  complex  decisions.  Decomposability  means  that  the  aspects  of 
consequences  relating  to  one  attribute  can  be  considered  independently  of  the  aspects 
of  consequences  relating  to  other  attributes. 

•  Operability:  The  operational  properties  are  concerned  with  obtaining  the  information 
useful  for  thinking  and  analysis.  The  model  should  be  understandable  to  everybody 
who  will  use  it.  Operability  is  an  issue  between  specialists  and  the  end  users.  The 
model  should  be  easily  explained  to  the  end  users  or  the  other  people  related  to  the 
process. 

•  Small  Size:  A  small  model  is  easy  to  use  and  understand.  Thus,  it  is  desirable  to  have 
smaller  hierarchies,  all  other  things  being  equal  (Kirkwood,  1997:18).  A  small  model 
is  also  considered  more  robust  when  compared  to  a  bigger  one.  If  the  given  decision 
is  repetitive  with  different  inputs,  then  robustness  may  become  a  critical  issue  for  the 
model. 


25 


A  complete,  nonredundant,  and  decomposable  model  forms  the  conjectural  background 
for  value  focused  thinking.  An  operable  and  small  hierarchy  helps  the  end  users  to 
provide  better  results  in  shorter  periods. 

2.6.23  Developing  Evaluation  Measures 

Evaluation  measures  are  used  to  rate  of  how  well  an  alternative  does  with  respect 
to  each  objective  (Keeney,  1992:100).  Evaluation  measures  form  the  x-axis  of  the 
metrics.  Time,  money,  and  number  of  peoples  are  some  common  evaluation  measures’ 
examples  used  in  many  studies.  Suppose  minimizing  the  loss  of  life  is  the  fundamental 
objective  of  a  study.  An  obvious  attribute  will  be  the  annual  number  of  fatalities. 
However,  it  is  not  easy  to  find  appropriate  evaluation  measures  for  some  studies. 
Different  evaluation  measures  may  be  needed. 

There  are  essentially  three  types  of  evaluation  measures: 

•  Natural  evaluation  measures:  Natural  attributes  have  a  common  interpretation  to 
everyone.  Profit  in  dollars  is  a  natural  evaluation  measure  for  many  business 
decisions. 

•  Constructed  evaluation  measures:  It  is  impossible  to  come  with  a  natural  evaluation 
measure  for  every  objective.  Examples  of  such  objectives  include  “increasing  the 
international  prestige  of  the  country”,  “improving  the  image  of  the  corporation”,  and 
“improving  the  morality  of  workers”.  Improving  the  morality  of  the  workers’ 
objective  can  be  quantified  in  an  evaluation  measure  by  using  three  different  levels: 

•  Bad  morality 
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•  Good  morality 

•  Great  morality 

•  Proxy  Attribute:  In  some  cases,  it  may  be  necessary  to  utilize  an  indirect  measure.  A 
proxy  evaluation  measure  reflects  the  degree  of  attainment  of  its  associated  objective, 
but  does  not  directly  measure  this  (Kirkwood,  1997:24).  The  overall  objective  for  an 
emergency  ambulance  system  might  be  to  “deliver  the  patience  in  the  best  possible 
conditions”.  Since  there  is  no  obvious  evaluation  measure  for  this  decision  problem, 
a  proxy  evaluation  measure  is  needed.  Stevenson  (1972)  has  used  the  proxy  attribute 
“response  time”  for  this  case. 

2.6.2.4  Single  Dimensional  Value  Functions 

The  single  dimensional  value  function  converts  an  evaluation  measure  into  value. 
Value  is  typically  measured  between  0  and  1  (Kirkwood,  1997:61).  The  scale  used  for 
analysis  has  no  effect  on  the  model  results  as  long  as  the  same  scale  is  used  for  all 
measures  in  the  value  model. 

Kirkwood  talks  about  two  types  of  value  functions: 

•  Piecewise  Linear  Functions 

•  Exponential  functions 

Both  functions  are  used  in  practical  applications.  The  piecewise  linear  function  is  easy  to 
use  when  the  evaluation  measures  have  a  small  number  of  possible  different  scoring 
levels.  The  piecewise  linear  single  dimensional  value  function  is  more  widely  used  by 
the  practitioners  of  VFT.  On  the  other  hand,  the  exponential  function  may  fit  the 
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preferences  of  the  decision-maker  properly  for  some  decision  problems.  Figure  2.4 
shows  the  examples  of  both  single  dimensional  value  functions. 


Figure  2.4  Piecewise  and  Exponential  Single  Dimensional  Value  Functions  Example 


Another  form  of  an  evaluation  measure  with  categories  is  called  a  discrete  value 
function.  Discrete  value  functions  use  discrete  scoring  levels  to  produce  values.  A 
special  case  of  a  discrete  function  is  a  binary  function.  An  alternative  either  gets  all  of 
the  value  (1)  or  none  of  the  value  (0)  for  the  function  in  this  situation. 


2.6.2. 5  Assessing  Weights 

Many  different  techniques  are  used  to  assess  the  model  weights.  Some  of  them 
are  anchored  rating  scales,  paired  comparisons,  and  direct  assignment  (Nutt,  1989:413). 
Each  weighting  technique  has  strengths  and  weaknesses. 

The  method  of  swing  weights  is  commonly  used  to  assess  the  weights  for  the 
values  in  a  hierarchy  although  other  methods  such  as  pricing  out  and  lottery  weights  are 
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values  or  measures  in  the  hierarchy  is  considered  individually.  The  weight  for  an 
evaluation  measure  is  equal  to  the  increment  in  value  that  is  received  from  moving  the 
score  on  that  evaluation  measure  from  its  least  preferred  level  to  its  most  preferred  level 
(Kirkwood,  1997:68).  This  property  provides  a  base  for  determining  the  weights. 
Kirkwood  provides  a  small  algorithm  in  his  book: 

•  Step  1:  Consider  the  increments  in  value  that  would  occur  by  increasing,  swinging 
each  of  the  evaluation  measures  from  the  least  preferred  end  of  its  range  to  the  most 
preferred  end,  and  place  these  increments  in  order  of  successively  increasing  value 
increments. 

•  Step  2:  Quantitatively  scale  each  of  these  value  increments  as  a  multiple  of 
smallest  value  increments. 

•  Step  3:  Set  the  smallest  value  increment  so  that  the  total  of  all  increments  is  1. 

•  Step  4:  Use  the  results  of  Step  3  to  determine  the  weights  for  all  the  evaluation 
measures  (Kirkwood,  1997:70). 

A  simple  example  presented  in  Figure  2.5  explains  the  swing  weights  process.  Assume 
that  you  are  trying  to  assign  weights  to  three  values  determined  to  be  important  when 
buying  a  car. 


Figure  2.5  Assessing  the  weights  example 
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The  value  increments  received  from  moving  the  score  of  an  evaluation  measure  from  its 
worst  level  to  its  best  level  while  the  other  evaluation  measures  are  held  constant  at  their 
worst  levels  are  used  to  find  the  weights  as  mentioned  previously.  In  this  case,  the 
decision-maker  may  tell  you  that  value  increment  created  by  changing  safety’s  score 
from  its  worst  level  to  its  best  level  is  three  times  as  important  as  comfort’s  value 
increment  for  the  fundamental  objective  and  value  increment  created  by  changing 
comfort’s  score  from  its  worst  level  to  its  best  level  is  two  times  as  important  as  image’s 
value  increment  for  the  fundamental  objective.  “X”  defines  the  smallest  value  increment 
in  this  example.  The  proportions  of  the  metrics’  weights  are  shown  in  Figure  2.6. 


Figure  2.6  Assessing  the  weights  example:  Mathematical  procedure 


The  weights  can  be  calculated  as  follows: 

We  know  that  they  have  to  add  up  to  “1”. 
6*X+2*X+X  =1  9*X=1  therefore,  X=0.1 11 

Thus  the  weights  are  shown  below: 

Safety  weight  =  0.666 
Comfort  weight  =  0.222 
Image  weight  =0.111 
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2.6.2. 6  Finding  Alternatives  and  Attribute  Scoring 


The  decision-maker  has  to  consider  two  key  points: 

•  You  can  never  choose  an  alternative  you  have  not  considered, 

•  Your  chosen  alternative  can  be  no  better  than  the  best  of  the  lot. 

The  VFT  approach  helps  create  new  alternatives.  In  this  way,  better  alternatives 
may  be  defined  for  the  problem. 

Kirkwood  (1997)  identifies  possible  methods  of  improving  or  finding 
alternatives.  He  suggests  considering  each  evaluation  measure  one  at  a  time  and 
identifying  ways  to  improve  the  alternative  in  that  particular  area.  It  may  be  that  the 
alternative  is  not  attractive  by  improving  it  in  a  single  area,  but  the  exercise  can  suggest 
other  attractive  alternatives. 

Scoring  alternatives  is  straightforward  but  can  be  time  consuming.  The  value 
scores  for  each  attribute  are  combined  using  value  model  weights  and  the  overall  value 
model  function.  The  alternative  receives  a  single  measure  of  merit  for  the  overall 
fundamental  objective  being  considered  for  the  decision. 

2.6.2.7  Ranking  of  Results 

The  additive  value  function  incorporates  weight  and  score  in  calculating  the 
overall  value  for  each  alternative.  A  rank  order  of  alternatives  can  then  be  accomplished 
based  on  these  values.  Kirkwood  provides  information  on  how  to  rank  results.  The 
graphical  techniques  used  during  the  process  provide  insights  about  the  decision  problem. 
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Kirkwood  provides  implementation  methods  to  determine  the  contribution  each  value 
makes  to  the  scoring  of  an  alternative  (Kirkwood,  1996:76-81). 

2.62.8  Sensitivity  Analysis 

Exploring  how  alternative  preferences  change  as  the  weights  assigned  to  the 
decision  criteria  shift  is  often  useful.  This  gives  the  decision-maker  a  justification  to  act 
and  provides  a  defense  for  actions  taken  (Nutt,  1989:480). 

Sensitivity  analysis  is  performed  by  changing  the  weight  of  a  single  evaluation 
measure  while  holding  other  weights  to  the  same  ratio  that  is  defined  by  the  decision¬ 
maker.  Since  all  the  weights  must  sum  to  1,  a  particular  weight  can  only  change  between 
0  and  1.  If  the  procedure  is  insensitive  to  meaningful  variations  in  the  weights,  further 
discussion  is  not  necessary. 

2.7  Resource  Allocation 

Resource  allocation  decision  problems  arise  whenever  there  are  special  funding 
patterns  and  new  projects  have  resource  requirements  which  exceed  available  ones. 
Bretschneider  (1993)  notes  that  constrained  optimization  models  have  a  set  of  equations 
containing  decision  variables  called  constraints.  The  models  also  contain  the  objective 
function,  and  decision  variables.  Linear  programming,  integer  programming  and 
nonlinear  programming  are  optimization  models  typically  used  in  resource  allocation 
decision  analysis. 
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Kirkwood  discusses  benefit/cost  analysis  and  optimization  methods  for  solving 
resource  allocation  problems  (Kirkwood,  1996: 199).  The  process  calculates  the  ratio  of 
the  benefit  of  the  project  to  the  cost  of  funding  the  project.  This  ratio  provides  insight 
based  on  the  benefit  per  dollar  invested.  There  are  some  problems  with  this  approach. 
The  model  can  handle  only  one  constraint,  and  most  practical  applications  involve  more 
than  one  constraint. 

Baker  and  Freeland  (1975)  identify  some  of  the  limitations  in  R&D  and  resource 
allocation  Models: 

•  Inadequate  treatment  of  risk  and  uncertainty 

•  Inadequate  treatment  of  multiple,  often  interrelated  criteria 

•  Inadequate  treatment  of  project  interrelationships  with  respect  both  to 
value  contribution  and  resource  utilization 

•  No  explicit  recognition  and  incorporation  of  the  experience  and 
knowledge  of  the  R&D  manager 

•  The  inability  to  recognize  and  treat  nonmonetary  aspects  such  as 
establishing  and  maintaining  balance  of  R&D  problem  (e.g.,  basic 
between  basic  and  applied  work,  between  product  and  process  effort,  and 
between  high  risk  high  payoff  and  moderate  or  low  risk  low  payoff 
opportunities) 

•  Perceptions  held  by  the  R&D  managers  hat  the  models  are  unnecessarily 
difficult  to  understand  and  use 
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•  Inadequate  treatment  of  the  time  variant  property  of  data  and  criteria  and 
the  associated  problem  of  consistency  in  the  research  program  and  the 
research  staff 

Baker  and  Freeland  conclude  that  R&D  and  resource  allocation  models  are 
incomplete  in  the  sense  that  they  do  not  include  all  the  important,  relevant  aspects  of  the 
R&D  environment.  As  a  result  the  manager  is  forced  to  adjust  the  recommended 
allocations  in  order  to  account  for  numerous  environmental  conditions  not  included  in  the 
model. 

2.8  Summary 

This  chapter  introduced  the  different  aspects  of  the  problem.  Safety,  combat 
capability,  and  the  cost  issues  were  explained.  It  reviewed  the  technology  selection 
models.  An  in  depth  review  of  VFT  and  AHP  were  provided.  Some  previous  VFT 
studies  were  examined.  Finally,  the  R&D  models  and  their  limitations  were  reviewed  to 
complete  the  chapter. 


34 


Chapter  3:  Methodology 


3.1  Introduction 

Different  types  of  R&D  technology  selection  models  were  discussed  in  Chapter  2. 
The  main  goal  of  this  study  is  to  select  the  best  set  of  modifications  for  F-16  capability 
enhancement.  Multidimensional  scoring  models  provide  the  most  appropriate  approach 
for  the  study  since  R&D  is  multidimensional.  Many  multidimensional  R&D  models  have 
been  used  in  the  past  for  technology  selection  problems  (Golabi,  Kirkwood,  Sicherman, 
1981).  Two  types  of  multidimensional  scoring  models  were  discussed  (VFT  and  AHP). 
VFT  was  the  best  fit  because  it  does  not  have  any  theoretical  problems  and  limitations 
known  to  exist  in  AHP.  VFT  also  allows  development  of  the  new  alternatives  (Keeney, 
1992:38-39)  and  does  not  require  reevaluation  of  all  its  alternatives  when  a  new  one  is 
added  to  the  model  (Bretschnedier,  1993:127-128). 

Chapter  3  begins  with  an  explanation  of  the  framework  for  the  study.  The  second 
topic  covered  is  the  method  of  developing  the  value  hierarchy.  A  modified  VFT 
approach  is  discussed  to  give  a  better  understanding  of  the  subject  in  that  section.  The 
model  is  divided  into  two  different  sections.  The  first  section  is  called  decision  weights. 
This  section  helps  decision-makers  modify  their  models  depending  on  their  preferences. 
The  second  section  is  the  value  hierarchy  model.  The  values  of  the  decision-maker  are 
explained  in  the  beginning  of  value  discussion.  The  development  of  the  evaluation 
measures,  weighting  the  values,  value  functions,  and  using  the  additive  value  function  are 
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the  final  parts  of  the  process  explained  in  the  second  topic.  Next  the  value  model  is 
merged  to  the  decision  weights  to  complete  the  process.  The  last  topic  covered  is  the 
assumptions  and  limitations  of  the  model.  A  summary  of  the  chapter  is  presented  to 
complete  the  chapter. 


3.2  Analysis  Process  and  Framework 

The  framework  identifying  analysis  steps  needed  for  modification  selection  in  F- 
16  model  is  shown  in  Figure  3.1.  This  section  summarizes  the  process.  The  first  step  in 
the  study  was  to  identify  a  value  hierarchy  for  F-16  aircraft,  called  the  F-16  Capability 
Enhancement  Model.  F-16  SPO  leaders  and  ASC  experts  were  consulted  to  confirm, 
modify,  and  develop  value  definitions.  Top-down  value  structuring  method  was  used  to 
build  the  model.  This  approach  defines  the  overall  objective  and  divides  the  overall 
objective  into  objectives.  Objectives  are  also  divided  into  sub-objectives  to  capture  the 
values  of  the  decision-maker  in  detail.  Such  an  approach  is  referred  to  as  “objectives- 
driven”  as  well  (Kirkwood,  1997:21).  The  final  model  represents  the  values  for  F-16 
program  in  the  Air  Force.  Certain  assumptions  and  limitations  of  the  model  will  be  stated 
in  detail  later. 

Evaluation  measures  were  developed  after  the  value  hierarchy  was  approved. 
Piecewise  linear  and  exponential  functions  were  developed  according  to  the  decision¬ 
maker’s  opinion. 


36 


F-16SP0 


F-16  Engine 
Modifications 


-Insights  for  the 
Leadership 
-Value  of  the 
modifications 


■Sensitivity 

Analysis 


Figure  3.1  Study  Framework 


The  evaluation  measures  combined  with  the  value  hierarchy  and  weights  to  define 
the  model.  Due  to  the  complexity  of  data  gathering,  it  was  difficult  and  slow  to  collect 
the  relevant  data  for  building  the  evaluation  measures. 

The  modifications  were  examined  as  a  starting  point  for  alternatives  in  this  study. 
Since  the  F-16  SPO  has  over  100  modifications,  a  small  subset  is  selected  for  analysis  of 
the  model.  Technical  experts  are  used  to  complete  this  part  of  the  study.  After 
identifying  alternatives,  each  alternative  is  scored. 

Each  alternative’s  score  is  changed  into  a  value  between  0  and  1  using  single 
dimensional  value  functions.  The  overall  fundamental  objective  score  is  then  calculated 
for  each  alternative.  Graphs  are  used  to  show  the  contribution  of  each  value  for  each 
alternative.  The  alternatives  are  ranked  according  to  their  overall  score  and  sensitivity 
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analysis  is  used  to  determine  how  changes  in  the  weights  affect  the  results  and  resulting 
decision  policy. 

Procurement  cost  is  used  to  identify  the  benefit  to  F-16  capability  per  dollar 
invested  for  each  modification.  A  new  ranking  of  the  alternatives  shows  the  change  in 
the  decision  policy  depending  on  the  money  constraint. 

Building  the  evaluation  measures,  alternative  scoring,  graphing  the  results,  and 
sensitivity  analysis  were  implemented  by  using  Microsoft  Excel  and  Logical  Decisions. 
Kirkwood’s  Excel  techniques  (Kirkwood,  1997)  and  some  macros  were  used  for  the 
automation  of  the  process. 

3.3  Method  for  the  Value  Hierarchy 

The  value  model  reflects  what  is  important  to  enhance  the  capability  of  the  F-16 
aircraft.  F-16  SPO  and  other  organizations  have  not  previously  used  a  systematic  process 
of  decision-making  like  VFT  or  AHP.  These  organizations  have  used  their  “values”  to 
make  decisions,  however  they  have  never  used  a  mathematical  model  to  quantify  them. 
The  process  of  decision-making  using  VFT  must  be  explained  at  the  beginning  of  the 
process  to  familiarize  people  with  multidimensional  scoring  models.  Neither  gold 
standard  nor  silver  standard  are  used  directly  in  this  study.  While  the  data  for  the  F-16 
model  is  available,  no  single  USAF  document  is  available  that  includes  all  values. 
Existing  literature,  documents  and  expert’s  opinions  are  used  together  to  fill  in  the 
information  gaps  in  the  model. 
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3.4  Building  the  Value  Hierarchy 

3.4.1  Introduction 

Like  many  decision  analysis  studies,  our  early  meetings  to  develop  and  refine  the 
value  model  were  time  consuming.  We  began  by  studying  the  literature  and  using  past 
experiences  to  build  the  model.  Problems  of  the  early  value  models  and  their  solutions 
can  be  found  in  Appendix  B.  It  is  possible  that  we  have  a  means  objective  model  instead 
of  an  ends  objective  model,  since  the  final  decision-maker  (ACC/DR)  could  not 
participate  in  the  model  development  process.  While  the  values  of  ACC/DR  might  differ, 
the  mean  objectives  are  important  because  they  lead  to  achievement  of  the  fundamental 
objective  to  enhance  the  capability  of  the  F-16  aircraft. 

A  modified  VFT  approach  was  accepted  as  the  best  solution  to  our  problems.  The 
next  section  explains  the  value  model  and  approach  in  detail. 


3.4.2  Building  the  Model 

Figure  3.2  shows  the  final  value  model.  We  show  the  model  two  different 
sections: 

•  Top  tier  (decision  weights) 

•  Value  hierarchy  model 

The  top  tier  helps  the  decision-maker  to  justify  the  model  for  the  changing  preferences  of 
the  decision-maker.  These  weights  can  be  used  for  many  different  purposes  due  to  the 
aircraft  block.  This  model  can  quantify  the  achievement  of  different  alternatives  for 
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various  purposes.  The  model  can  change  the  benefits  of  alternatives  for  altering  decision 
weights  due  to  modified  aircraft  blocks.  Appendix  B  explains  the  purpose  of  decision 
weights  in  detail. 


Figure  3.2  Final  Model  and  Modified  VFT  Approach 


The  data  for  modifications  that  affect  multiple  blocks  are  divided  into  the  data 
specific  for  each  individual  block.  The  model  quantifies  each  specific  block  data.  An 
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engine  score  for  each  block  type  helps  identify  the  contribution  each  block  makes  to  the 
overall  engine  modification.  The  next  step  is  assigning  the  block  weights  (decision 
weights)  of  the  top  tier.  These  weights  are  assigned  by  the  decision-maker.  Multiplying 
the  weights  with  the  total  scores  for  each  block  gives  the  overall  block  value.  These 
block  values  are  added  together  to  find  an  overall  alternative  score.  Ranking  is  the  final 
step  in  the  model.  A  simple  example  in  Figure  3.3  describes  the  process  better. 


Figure  3.3  Modified  VFT  Approach  Example 


The  modification  data  is  broken  into  the  specific  block  type  data  (e.g. 


modification  A(30),  modification  A(40),  etc.)  for  each  block  type  affected  by  the 
modification.  The  data  is  run  through  the  model  to  get  the  final  scores,  fl,f2,  etc..  The 
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decision-maker  assigns  the  decision  weights  for  the  top  tier  (wl,w2...  etc.).  The  overall 
value  can  be  used  for  different  ranking  processes,  wX*fX  for  specific  block  type  and 
X  wX*fX  for  the  overall  value  of  modifications. 

The  accuracy  and  flexibility  of  the  model  enabled  the  decision-maker’s  opinions 
about  the  aircraft  blocks  to  be  blended  with  the  analytical  methods  of  VFT.  We  accept 
this  approach  as  the  solution  to  the  problems. 

3.5  Model  Definitions 

Building  the  model  was  the  most  critical  part  of  the  study.  The  results  are  based 
on  multiple  meetings  and  long  discussions  about  “values”.  The  model  is  not  intended  to 
be  a  means-objective  model.  However,  there  may  be  certain  values  that  cannot  be 
captured  or  clearly  defined  due  to  time  and  decision-maker  constraints. 

The  overall  value  (the  fundamental  objective)  is  selecting  the  best  set  of  best 
engine  modifications  to  enhance  F-16  capability.  Best  is  defined  as  having  a  higher 
model  score  and  having  a  bigger  ratio  in  value/cost  analysis  when  compared  to  others 
modifications.  Only  engine  modifications  are  considered  due  to  time  and  data  constraints 
involved  in  evaluating  all  modifications.  However,  the  study  proved  that  VFT  can  be 
applied  to  ACC’s  decision-making  process  as  a  support  system. 

The  overall  value  was  divided  into  three  objectives  as  shown  in  Figure  3.4.  The 
bold  portion  of  the  model  identifies  the  current  section  being  discussed. 
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Figure  3.4  Objectives  of  the  Model 

First,  the  combat  capability  objective  of  the  value  model  is  explained.  Next,  sub¬ 
objectives  are  discussed  under  combat  capability  objective.  Evaluation  measures  of 
combat  capability  are  introduced  to  fill  the  meaning  of  the  combat  capability  as  a  value  of 
the  decision-maker.  Simple  examples  of  some  discussions  help  to  understand  the  true 
meaning  of  the  objective.  Safety  impact  and  operational  costs  are  also  explained  in  the 
same  manner  respectively. 


3.5.1  Combat  Capability 

Combat  capability  is  considered  to  be  a  heavily  weighted  part  of  the  model. 


43 


The  intent  of  this  objective  is  to  capture  the  increased  combat  capability  that  the 
modification  provides  for  the  F-16  aircraft. 

As  explained  earlier,  achieving  air  superiority  and  conducting  precision  attacks 
are  the  key  elements  in  fighting  and  winning  the  war  (The  Nation’s  Air  Force,  2000). 
Achieving  air  superiority  and  conducting  precision  attacks  provide  the  base  for  combat 
capability.  Representatives  from  F-16  SPO,  AFIT,  and  ASC  worked  as  a  decision  team. 
The  team  tried  to  concentrate  on  these  two  goals  for  the  study.  However,  the  discussion 
about  what  they  mean  to  F-16  aircraft  was  limited  in  scope  due  to  the  time  and  data 
constraints. 

The  team  came  up  with  two  different  statements  to  capture  the  importance  of  air 
superiority  and  precision  attacks: 

•  Aircraft  must  be  ready  for  takeoff  on  the  runway. 

•  The  goal  is  to  fly  the  mission,  kill  our  enemy’s  fighters  in  air-to-air  combat, 
bomb  the  targets,  and  fly  back  to  the  base. 

The  combat  capability  objective,  as  shown  in  Figure  3.5,  was  divided  into  two 
different  sub-objectives  under  the  lights  of  these  two  statements: 

•  Aircraft  Availability 

•  Combat  Score 
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Figure  3.5  Combat  Capability 


3.5. 1.1  Aircraft  Availability 

The  aircraft  availability  model  used  by  ACC  was  not  the  model  used  for  the 
study.  Instead,  a  simple  but  effective  metric  was  needed  to  find  the  availability  of  aircraft 
to  carry  out  the  mission. 

Studies  and  experience  show  that  unscheduled  engine  removals  (UER)  is  a 
valuable  and  effective  metric  for  the  value  hierarchy  model.  Interviews  with  engine 
maintenance  personnel  show  that  the  biggest  impact  for  aircraft  availability  comes  from 
UER  drivers,  as  a  UER  process  took  about  16-hours  (Bullerman,  2000).  Other 
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maintenance  procedures  are  mainly  scheduled  and  short  processes.  Therefore,  UER  data 
was  selected  as  a  good  metric  for  the  study.  This  metric  dominated  all  others. 

3.5. 1.2  Combat  Score 

Combat  score  is  intended  to  capture  the  value  of  conducting  precision  attacks. 
Combat  score  is  the  most  important  sub-objective  in  the  model.  The  main  goal  for 
modernizing  fighter  aircraft  is  to  keep  their  combat  value  up  to  date.  However,  it  is 
assumed  that  engine  modifications  will  not  have  a  big  impact  on  this  sub-objective.  The 
engine  modifications  are  focused  on  fixing  or  removing  documented  defects  that  decrease 
flight  safety.  Therefore,  combat  score  is  briefly  discussed,  but  does  not  have  a  weight  in 
the  model  due  to  the  scope  of  this  research.  However,  three  different  metrics  were 
defined  for  the  completeness  of  the  model: 

•  Kill  Ratio:  The  kill  ratio  is  an  air-to-air  combat  metric  that  measures  the  number 
of  enemy  aircraft  that  are  destroyed  per  USAF  aircraft  loss.  Kill  ratio  depends  mainly  on 
the  weapon  system  capabilities  of  the  fighter.  Smart  air-to-air  weapon  systems  have 
increased  the  kill  ratio  of  the  F-16  in  recent  years.  The  F-16  air  superiority  capabilities 
were  proven  in  the  Gulf  War.  One  of  the  main  reasons  for  building  the  F-16  was  to 
improve  on  the  poor  kill  ratio  of  former  fighter  aircraft  like  the  F-4.  Increasing  kill  ratio 
is  one  of  the  main  drivers  for  ACC  to  enhance  the  capability  of  the  F-16  fighter. 

•  Circular  Error  Probability  (CEP):  CEP  is  an  air  to  ground  metric  that  measures 
bombing  accuracy  with  lower  CEP  indicating  a  more  accurate  weapon  system.  CEP  is 
defined  as  the  radius  from  target  into  which  a  munition  can  be  placed  at  least  half  the 


46 


time  (Smart  Munitions,  2000).  This  value  is  also  a  part  of  the  weapon  system.  Smart 
weapon  systems  are  designed  to  decrease  the  CEP  and  improve  bombing  accuracy  for  the 
fighter  aircraft.  It  is  one  of  the  main  considerations  for  ACC.  The  radar  guided  weapon 
systems  in  F-16  increases  its  bombing  accuracy  and  permits  the  pilot  drop  bombs  in  bad 
weather  conditions  without  even  seeing  the  target.  The  system  collects  the  relevant 
weather  data  (like  wind)  and  guides  the  weapon  system  due  to  data  changes.  This 
capability  makes  aircraft  a  perfect  air-to-ground  weapon  system.  Based  on  these  reasons, 
the  modernization  of  air  to  ground  weapon  system  is  an  important  concern  for  ACC. 

•  Survivability:  Survivability  is  the  increased  capability  that  a  modification 
provides  to  prevent  a  combat  loss.  Survivability  features  include: 

•  Small  size 

•  Small  trace  on  the  radar  screen 

•  High  sustained  speeds 

•  High  agility 

•  Situation  awareness  features 

•  Countermeasures  equipment 

•  Buried  fuel  lines 

•  Fuel  inerting 

•  Critical  systems  redundancy  and  shielding 

•  Rugged  nine-g  structure  with  alternate  load  paths 

The  F-16  has  lethal  self-defense  against  air  threats  with  features  such  as  radar,  guns,  all¬ 
aspect  air-to-air  missiles,  electronic  warfare  suites,  and  towed  decoys  (Fighter  Programs, 
2000). 


3.5.2  Safety  Impact 

Safety  impact  is  considered  the  second  most  important  concern  in  the  model.  The 
goal  is  to  quantify  the  effect  of  a  particular  modification  on  flight  safety.  The  question 
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was,  “What  was  the  best  measure  for  capturing  the  impact  to  flight  safety?”  Detailed 
studies  point  to  Class  A  mishap  rates  (per  100K  flight  hours)  as  the  best  measure  for  the 
risk  evaluation. 

The  mishap  classes  are  labeled  A,  B,  and  C.  Class  A  refers  to  mishaps  resulting 
in  fatality,  destroyed  aircraft  or  more  than  $1  million  in  damage.  Class  B  includes  those 
resulting  in  permanent  partial  injury  or  more  than  $200,000  in  damage.  Class  C,  the  most 
common  class  of  mishap,  are  those  resulting  in  hospitalization  or  more  than  $10,000  in 
damage  (Air  Force  Instruction  91-204,  29  Nov  1999).  Class  A  mishap  rate  is  an  effective 
measure  because  it  captures  the  values  of  the  decision-maker,  it  is  easily  derivable,  and  it 
is  a  well-known  value  in  the  Air  Force  organizations. 

Another  important  aspect  of  the  safety  benefit  of  a  modification  is  the  time  it 
takes  to  incorporate  the  modification  into  the  fleet.  Installation  schedule  for  the 
modifications  is  used  to  capture  our  concerns  for  the  time.  A  simple  example  outlined  in 
Table  3.1  better  explains  the  problem. 


Table  3.1  An  Example  for  Installation  Schedule 


Modification 

Installation  Year 

Installation  Year 

Installation  Year 

Installation  Year 

Type 

Year  1 

Year  2 

Year  3 

Year  4 

A 

250  Aircraft 

100  Aircraft 

0  Aircraft 

0  Aircraft 

B 

100  Aircraft 

100  Aircraft 

100  Aircraft 

50  Aircraft 

C 

0  Aircraft 

0  Aircraft 

50  Aircraft 

300  Aircraft 

As  shown  in  Table  3.1,  each  modification  impacts  350  total  aircraft.  The  decision¬ 
makers  (ASC  and  F-16  SPO  representatives)  ranked  the  modification  A-B-C  citing  a 
desire  to  modify  as  many  aircraft  as  possible  in  the  early  years  of  the  project.  There  are 
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several  reason  reasons  driving  this  factor.  One  reason  is  the  uncertainty  about  the  future; 
another  is  political  concerns.  Modifying  a  specific  number  of  each  engine  type,  General 
Electric  (GE)  and  Pratt  Whitney  (PW),  depending  on  the  future  use  of  the  aircraft  is  a 
political  decision.  However,  the  most  important  issue  is  the  Air  Force  losing  jets  every 
flight  year  due  to  well-known  problems,  and  these  modifications  reduce  mishap  rates. 
The  team  decided  that  the  best  case  is  modifying  all  the  aircraft  in  the  first  year. 
However,  this  is  not  always  possible  due  to  constraints  like  money,  man-hours  ...  etc.  A 
new  approach  is  needed  to  include  time  considerations  in  our  model. 

Considering  these  different  aspects  of  the  problem,  we  divide  safety  into  two 
different  sub-objectives  as  shown  in  Figure  3.6: 

•  Total  Class  A  mishap  rate  change 

•  Installation  schedule 
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Figure  3.6  Safety  Impact 


3. 5.2.1  Total  Class  A  Mishap  Rate  Change 

Total  Class  A  Mishap  Rate  Change  is  designed  to  capture  the  decrease  in  the 
mishap  rate  that  the  modification  would  provide.  Two-  different  metrics  are  used  for 
quantifying  the  alternatives: 

•  Number  of  Class  A’s  (Initial-Threshold/Final , whichever  is  greater) 

•  Number  of  Class  A’s  (Baseline-Threshold/Final , whichever  is  greater) 

Figure  3.7  helps  the  reader  to  understand  these  two  metrics  and  correlate  with  the 

definitions  provided. 
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Figure  3.7  Risk 
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A  several  definitions  help  to  clarify  the  example. 

•  Initial  risk:  This  is  the  risk  value  before  inspection  and  installation  of  the 
modification  (unmitigated  risk). 

•  Baseline  (Mitigated)  risk:  This  is  the  value  of  risk  after  inspection.  If  the 
inspection  is  done  an  infinite  number  of  times,  hypothetically  you  can  reduce  the  risk  to  a 
value  close  to  0.  This  is  impossible  due  to  constraints  like  money,  man-hours  ...etc. 
Therefore,  the  number  of  part  inspections  are  traded  off  against  money  and  time. 

•  Threshold  risk:  This  value  is  defined  by  F-16  SPO  for  different  engine  failure 

types. 

•  Final  Risk:  This  is  the  value  after  modification  installed  to  the  aircraft. 

Inspection  is  a  part  of  the  process. 

After  understanding  these  definitions,  the  metrics  were  constructed  to  quantify  the 
alternatives. 
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The  first  metric,  Initial-Threshold/Final  whichever  is  greater,  captures  the  total 
difference  in  risk  by  Class  A  accidents.  The  bigger  the  difference,  the  more  important  the 
modification  is  to  the  F-16  community.  The  team  answered  the  question  of  why  initial 
risk  was  important  easily.  A  big  initial  risk  means  that  aircraft  are  being  lost  due  to  that 
specific  problem.  To  fix  the  problem  the  modification  should  have  a  greater  value. 

The  second  metric,  Baseline-Threshold/Final  whichever  is  greater,  gives 
information  about  the  difference  between  the  inspected  and  the  final  value  of  Class  A 
accidents.  One  of  the  main  concerns  in  that  area  is  how  to  handle  the  inspection  issue.  If 
hypothetically,  the  aircraft  can  be  inspected  an  infinite  number  of  times,  the  risk  value  is 
close  to  0.  However,  this  obviously  is  not  an  option.  Increasing  the  inspection  rate 
increases  cost,  and  is  time  consuming.  Therefore,  the  current  number  of  inspections  and 
mitigated  values  are  used  as  the  baseline.  Another  point  the  team  tried  to  capture  is  the 
mitigated  risk  being  equal  to  the  threshold.  The  ASC  and  F-16  SPO  are  not  willing  to 
choose  the  modifications  for  safety  reasons  alone  in  that  case.  The  bigger  the  difference, 
the  better  the  modification  because  there  was  more  risk  that  was  not  being  mitigated  by 
inspections. 

There  is  only  one  important  question  left  to  answer  to  justify  the  metrics:  “  Why 
did  we  want  to  use  Threshold/Final,  whichever  is  greater?” 

A  lot  of  questions  were  asked  to  the  decision-maker  during  this  process.  They  are 
basic  why  and  lottery  questions.  One  of  the  points  during  those  conversations  is 
especially  surprising.  The  decision-maker  said  that  the  difference  between  final  value 
and  threshold  was  not  important.  If  a  final  risk  value  is  smaller  than  the  threshold,  how 
much  you  are  under  the  threshold  does  not  affect  the  modification  selection  process  at  all. 
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If  a  final  risk  value  is  larger  then  the  threshold,  it  is  due  to  the  other  considerations 
beyond  the  modification  capability.  Thus,  the  word  greater  is  used  to  handle  these 
situations. 

Another  point  should  be  clarified  to  help  future  studies.  The  decision-maker  was 
asked  if  these  two  different  modifications  were  equally  preferred  without  considering  the 
initial  and  final  value.  The  answer  was  yes  as  they  were  concerned  with  the  total  change 
in  risk.  This  would  require  a  new  metric  to  be  developed  for  capturing  the  final  risk 
value.  However,  the  team  believed  that  our  metrics  were  adequate  to  cover  the  needs  of 
this  study.  Future  researchers  should  further  study  the  relationship  between  final  risk  and 
the  threshold  value  in  more  detail.  Figure  3.8  explains  this  issue  with  a  simple 
demonstration. 


A  N 
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Threshold 


Modification  A 
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Figure  3.8  A  Comparison  Example 


In  this  case,  A>C  and  B>D  and  A-B  =  C-D. 
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3. 5.2.2  Installation  Schedule 


The  time  to  complete  retrofit  of  the  modification  and  the  number  of  aircraft 
affected  was  captured  by  the  installation  schedule.  The  model  is  expected  to  penalize 
long  projects  due  to  problems  mentioned  previously.  Further  studies  are  needed  to  get  a 
better  metric  for  handling  this  challenging  issue. 

The  team  agreed  that  a  modified  net  present  value  approach  would  be  useful. 

This  new  method  is  called  weighted  annual  percentage  method  (WAPM). 

There  are  two  main  problems: 

•  The  number  of  aircraft  modified  per  year 

•  The  installation  year 

The  best  and  worst  case  hypothetical  examples  help  to  get  better  insights  into  the 
problem.  Table  3.2  shows  hypothetical  cases  where  the  number  of  total  aircraft  modified 
is  X  and  the  installation  schedule  is  Y  years: 


Table  3.2  Hypothetical  Best  and  Worst  Cases 


Modification 

Year  1 

Year  2 

.  YearY 

Modification  A  (The  Best  Case) 

X 

0 

0 

Modification  B(The  Worst  Case) 

0 

0 

X 

This  table  is  not  enough  to  solve  the  problem  as  neither  X  nor  Y  are  fixed  values. 
The  first  year  is  the  most  important  one,  therefore,  a  factor  of  lis  chosen  for  the  first  year. 
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The  other  years  take  smaller  values  such  as  Year  2  has  a  factor  of  0.9,  Year  3  has  a  factor 
of  0.8  and  so  on.  The  normalization  for  the  aircraft  number  helps  to  get  a  score  for  this 
specific  attribute.  Caution  is  needed  at  this  point,  because  value  function  should  be 
between  0  and  l(Keeney,  1992).  The  value  functions  are  needed  and  included  the 
weighted  annual  percentages  on  the  x-axis.  A  simple  example  shown  in  Table  3.3  helps 
explain  the  metric. 


Table  3.3  An  Installation  Schedule  Example:  Aircraft  Numbers 


Modification 

Year  1 

Year  2 

Year  3 

Year  4 

Total 

A 

200 

20 

40 

400 

660 

B 

200 

35 

50 

- 

285 

C 

50 

100 

- 

- 

Table  3.3  shows  660  aircraft  modified  in  4  years  in  the  first  case.  The  second 
case,  285  aircraft  are  modified  in  3  years.  The  third  modification  is  the  shortest  one,  only 
2  years  with  150  aircraft  modified.  Finding  the  best  modification  cannot  be  answered 
easily.  The  time  and  number  of  aircraft  modified  per  year  should  be  considered.  Table 
3.4  shows  the  year  factors  given  by  the  decision-maker. 


Table  3.4  Factoring  the  Years 


Years 

1 

2 

3 

4 

Factors 

1 

0.5 

0.25 

0 

These  factors  reduce  the  benefit  of  longer  projects.  If  the  project  is  longer  than  3  years, 
no  value  is  added  to  improve  the  alternative’s  score.  An  infinite  number  of  year  factors 
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can  be  found.  However,  the  factors  are  determined  by  the  decision-maker’s 
representative  and  will  be  analyzed  during  sensitivity  analysis. 

Normalization,  as  shown  in  Table  3.5,  is  the  step  before  the  final  result. 
Normalizing  captures  the  modified  aircraft  numbers  per  year  relative  to  the  number  of  all 
aircraft  modified. 


Table  3.5  Normalization  Values 


Modifications 

Normalization 
Value  For  Year  1 

Normalization 
Value  For  Year  2 

Normalization 
Value  For  Year  3 

Normalization 
Value  For  Year  4 

A 

200/660 

20/660 

40/660 

400/660 

B 

200/285 

35/285 

50/285 

- 

C 

50/150 

100/150 

- 

- 

The  final  result  is  demonstrated  in  Table  3.6.  The  overall  values  for  years  is  the 
multiplication  of  (modified  aircraft  number/total  aircraft  number)  times  the  year  factors. 


Table  3.6  The  Results  of  Installation  Schedule  Example 


Modifications 

Values  for 
Year  1 

Values  for 
Year  2 

Values  for 
Year  3 

Values  for 
Year  4 

Total 

A 

0.3*1=  0.3 

0.03*0.5  = 
0.015 

0.06*0.25= 

0.015 

0.6*0=  0 

0.33 

B 

0.7*1=  0.7 

0.12*0.5  = 
0.06 

0.18*0.25= 

0.04 

0 

0.8 

C 

0.33*1  = 
0.33 

0.66*0.5  = 
0.33 

0 

0 

0.66 

The  result  of  weighted  annual  percentage  technique  was  acceptable  for  this  case. 
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Modification  B  was  preferred  to  Modification  C  and  A.  Note  that  the  weights  and 
the  values  in  this  process  are  separate  from  the  VFT  process.  These  calculations  help  to 
develop  a  good  metric  for  the  installation  schedule.  However,  sensitivity  analysis  on  this 
technique  provides  better  insights  about  the  weighting. 


3.5.3  Operations  Costs 

Cost  is  a  major  concern  in  this  study.  Operations  costs  are  those  incurred  during 
the  project-life.  Operations  costs  are  divided  into  two  sub-objectives  as  shown  in  Figure 
3.9: 

•  Reduction  in  total  ownership  cost  (RTOC) 

•  Non-RTOC  costs 
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Score  Engine 
Mods  by  Block 


(greater  of  T  or  F) 


Figure  3.9  Operations  Cost 


3.5.3. 1  RTOC  Costs 


Reduced  total  ownership  cost  (RTOC)  is  briefly  explained  to  help  explain  the  sub¬ 
objectives  and  metrics  used  for  the  operations  cost.  These  two  sub-objectives,  RTOC  and 
non-RTOC  factors,  may  not  include  all  kinds  of  cost  considerations.  However,  studying 
total  ownership  cost  (TOC)  and  RTOC,  shows  that  a  big  part  of  cost  consideration  for  an 
Air  Force  project  can  be  captured  in  an  RTOC  model.  Non-  RTOC  costs  to  include  the 
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other  main  cost  drivers  in  the  model.  Under  Secretary  of  Defense,  Jacques  S.  Gansler 
points  out  the  importance  of  the  RTOC  model  for  the  Air  Force. 


We  are  facing  an  unprecedented  challenge  to  modernize  our  forces  in  a  world 
that  demands  more  efficient  as  well  as  more  effective  acquisition.  To  meet  that 
challenge,  we  need  to  take  the  next  big  acquisition  reform  step— the  Revolution  in 
Business  Affairs. 

For  this  next  phase  of  acquisition  reform,  we  must  further  adapt  the  best  world 
class  business  and  technical  practices  to  our  needs,  rationalize  our  infrastructure, 
restructure  our  support  systems,  and  reduce  cycle  times  and  ownership  costs.  The 
Defense  System  Affordability  Council  (DSAC)  is  our  forum  for  setting  and  monitoring  top 
level  goals,  objectives,  and  metrics  for  these  areas— metrics  which  must  be  mirrored  in 
each  and  every  DoD  acquisition  organization,  whether  it  be  a  program  office  acquiring  a 
new  system  or  a  logistic  organization  supporting  a  fielded  system. 

The  Honorable  Jacques  S.  Gansler, 

Under  Secretary  of  Defense  ( Acquisition  and  Technology) 


TOC  and  RTOC  concepts  are  designed  to  restructure  the  support  systems  for 
making  them  more  effective  and  efficient. 

DoD  TOC  is  comprised  of  costs  to  research,  develop,  acquire,  own,  operate, 
train,  and  dispose  of  weapon  and  support  systems,  other  equipment  and  real  property,  the 
costs  to  recruit,  train,  retain,  separate  and  otherwise  support  military  and  civilian 
personnel,  and  all  other  costs  of  business  operations  of  the  DoD  (DESAC  Strategic  Plan 
99). 

In  the  early  years  of  the  project,  O&S  costs  are  low  and  return  high  modernization 
values.  In  the  process,  costs  increase  while  return  modernization  value  decreases  to  get 
the  same  amount  of  benefit.  The  modernization  provided  for  the  same  amount  of  money 
begins  to  decrease  in  the  later  years  of  the  project.  Figure  3.10  shows  the  relationship 
between  the  modernization  and  cost. 
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TIME 


Figure  3.10  Modernization  vs.  Costs 


The  RTOC  model  helps  the  decision-maker  deal  with  the  increased  O&S  costs  in 
the  later  years  of  the  project.  Sun  Tzu’s  words  about  TOC  are  enlightening.  Why  is 
TOC  important  and  should  be  studied?  Why  must  it  be  controlled  and  reduced? 

As  to  government  expenditures,  those  due  to  broken  down  chariots,  worn-out 
horses,  armor  and  helmets,  arrows  and  crossbows,  lances,  hand  and  body  shields, 
draft  animals  and  supply  wagons  will  amount  to  60%  of  the  total. 

Sun  Tzu,  The  Art  of  War,  6th  Century  B.  C 

The  RTOC  program  is  the  result  of  longstanding  concern  about  impact  of 
declining  procurement  funds,  aging  inventory  and  continuing  high  operations  / 
deployment  levels  (Dr.  Spiros  Pallos,  RTOC  best  Practices  and  Lessons  Learned  Slides). 
Some  of  the  strategic  RTOC  plans  are: 

•  Investment  Strategies 

•  Effective  requirements  determination  processes 

•  Implementation  of  policies  to  reduce  TOC 
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Air  Force  uses  the  RTOC  cost  model  to  handle  the  increased  O&S  costs  of  the 


older  systems.  Air  Force  RTOC  primary  objectives  are: 

•  Cost  control 

•  Cost  reduction 

•  Invest  to  modernize 


(F-16  Offsite  RTOC  Briefing) 

After  understanding  the  importance  of  the  RTOC  cost  model,  the  team  added  two 
metrics  for  capturing  this  value: 

•  Change  in  the  cost  of  the  parts 

•  Change  in  the  cost  of  the  labor 

These  two  metrics  were  accepted  as  the  important  drivers  which  make  total  cost. 
Using  these  metrics  is  also  critical  for  the  model  credibility  since  this  study  was  the  first 
step  and  RTOC  cost  model  is  drawing  the  attention  of  the  senior  leaders. 


3.5. 3.2  Non-RTOC  costs 

The  other  sub-objective  is  the  Non-RTOC  costs.  Non-RTOC  costs  are  those  not 
included  in  the  RTOC  cost  model.  The  most  important  non-RTOC  metrics  were  O&S 
inspect  time  and  O&S  repair  time.  However,  maintenance  man  hours  (MMH)  was  used 
as  a  proxy  metric  to  cover  non-RTOC  costs.  MMH  includes  inspection  time  and  repair 
time  for  a  maintenance  problem  during  the  life  cycle  of  a  part.  One  main  discussion 
about  this  metric  was  whether  to  take  the  total  time  for  MMH,  or  to  use  the  longest  time 
period.  The  total  time  is  the  sum  of  separate  MMH  required  for  fixing  the  problems.  The 
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longest  one  is  the  maximum  MMH  required  for  fixing  any  problem.  Longest  time  period 
is  more  critical,  since  it  affects  the  F-16  turn  around  time.  It  was  agreed  that  tum  around 
time  was  more  important  because  it  was  affecting  the  time  on  target  (TOT).  TOT 
accuracy  is  accepted  as  the  most  important  value  for  the  war  fighter. 

3.6  Single  Dimensional  Value  Functions 

Armed  with  an  understanding  of  each  evaluation  measure,  we  present  the 
procedure  for  determining  the  single  dimensional  value  functions  used  in  this  study.  The 
range  for  each  evaluation  measure  depends  on  the  historical  data.  Value  functions 
represent  the  decision-maker’s  opinions.  Decision-makers  for  this  process  were 
Maj.Kricker  (F-16  SPO)  and  Mr.Hanke  (ASC  representative).  Two  different  software 
packages  were  used  for  this  part  of  the  study.  The  first  one  was  Microsoft  Excel. 
Kirkwood  techniques  were  implemented  using  Excel  Software  (Kirkwood,  1997). 

Logical  Decisions  was  used  for  comparing  the  results  during  verification.  The  full  data 
set  can  be  found  in  Appendix  D. 

3.6.1  UER  Drivers 

Historical  data  shown  in  Table  3.7  was  used  to  build  this  value  function. 
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Table  3.7  UER  Drivers 


UER  Driver 

Rate/ 

Fleet 

1000 

EFH 

Stalls 

0. 

071 

-229 

FOD/DOD 

0. 

062 

-229 

Oil  Leaks 

0. 

060 

-229 

Turbine  Nozzle/Blade  Failure 

0. 

051 

-100 

Augmentor  Liner  Deterioration 

0. 

038 

-229 

Cracked/Warped  Inlet  Guide  Vanes 

0. 

036 

-100 

Flameholder/Fuel  Ring  Damage 

0. 

030 

-100 

Turbine  Nozzle  Cracks/Failure 

0. 

027 

-129 

Turbine  Blade/Vane  Burn-Through 

0. 

022 

-229 

Damaged/Cracked  Turbine  Frame 

0. 

018 

-129 

Component  System  Malfunctions 

0. 

018 

-100 

Broken  Safety  Wire 

0. 

009 

-129 

Turbine  Section  Deterioration 

0. 

006 

-129 

Turbine  Blade/Vane  Burn-Through 

0. 

006 

-129 

Combustor  Damage 

0. 

006 

-100 

The  data  represents  the  occurrences  of  the  UER  drivers  for  specific  engine  types. 
However,  they  are  not  modified  for  aircraft  type.  This  study  assumes  that  the  figures  are 
calculated  for  engines  used  in  F-16.  The  decision-maker  expressed  that  a  linear  line  was 
appropriate  for  this  evaluation  measure  as  shown  in  Figure  3.1 1.  The  range  varies  from 
zero  to  0.07 1 .  Zero  was  added  to  the  range  because  some  alternatives  did  not  attack  the 
UER  drivers. 
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Figure  3.11  UER  Drivers 


3.6.2  Number  of  Class  A’s  (Initial  -  Threshold/Final  whichever  is  greater) 

The  range  for  this  evaluation  measure  is  from  zero  to  0.7.  The  historical  data  and 
decision-maker’s  experience  were  used  for  the  range.  The  decision-maker  explained  that 
up  to  a  value  of  0.1,  safety  was  not  a  big  concern.  However,  after  0.1,  safety  was 
becoming  an  important  issue  up  to  0.7.  He  was  more  concerned  with  the  safety  if  the 
score  became  bigger.  This  attitude  shows  that  an  exponential  curve  was  necessary  for 
that  part  of  the  range  as  seen  in  Figure  3.12. 
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Figure  3.12  Number  of  Class  A’ s  (Initial-Threshold/Final) 


3.6.3  Number  of  Class  A’s  (Mitigated  -  Threshold/Final  whichever  is  greater) 

The  range  for  this  evaluation  measure  varies  from  zero  to  0.7  as  well.  This  range 
matches  the  later  metric  because  for  some  modifications,  initial  values  and  mitigated 
values  are  the  same.  Safety  experts  constructed  the  mitigated  values  for  these  evaluation 
measures. 

The  attitude  of  the  decision-maker  is  similar  to  the  first  safety  evaluation  measure 
explained  above.  A  simple  line  is  used  up  to  0.1.  However,  since  this  was  mitigated  risk, 
the  decision-maker  was  more  concerned  about  the  difference.  Therefore  a  slightly 
steeper  exponential  function  is  used  as  shown  in  Figure  3.13. 
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Figure  3.13  Number  of  Class  A’s  (Initial-Threshold/Final) 


3.6.4  Weighted  Annual  Percentages 

The  method  explained  earlier  in  Chapter  3  is  used  to  get  the  x-axis  values  for 
WAPM.  The  year  factors  were  determined  and  every  year  after  5  gets  a  factor  0.  The 
factors  are  1  for  year  1, 0.8  for  year  2,  0.6  for  year  3,  0.4  for  year  4,  and  0.2  for  year  5. 
Year  factoring  values  reflect  the  decision-maker  preferences.  The  calculations  for 
WAPM  can  be  found  in  the  Appendix  D. 

The  range  for  the  WAPM  in  evaluation  measure  varies  from  0.4  to  1.0.  The 
decision-maker  expressed  that  everything  less  than  0.4  on  the  x-axis  was  equal  to  a  value 
of  zero.  An  upper  bound  of  1.0  is  used  because  it  is  possible  to  install  a  modification  to 
all  aircraft  in  the  first  year  (hypothetical  best  case  in  Chapter  2).  Figure  3.14  shows  the 
value  function  for  weighted  annual  percentage.  The  function  is  linear  line  since  the 
decision-maker  preferences  are  captured  in  year  factors. 
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Figure  3.14  Weighted  Annual  Percentages 


3.6.5  Delta  O&S  Parts 

This  evaluation  measure  has  a  range  from  $-1359  to  $100,000.  Negative  values 
represent  the  money  that  will  be  spent  on  the  project.  The  decision-maker  expressed  that 
the  savings  of  the  parts  were  insignificant  up  to  $16,000.  After  this  point  however,  he 
said  his  values  were  increasing  exponentially.  An  exponential  function  is  considered 
appropriate  to  build  the  value  function.  However,  Kirkwood’s  exponential  function  does 
not  work  due  to  the  range  of  the  data  (Kirkwood,  1997:65).  The  data  had  to  be  modified. 
The  first  step  involves  gathering  the  entire  data  set.  If  the  minimum  value  is  negative,  the 
entire  data  set  is  shifted  by  this  amount  to  ensure  positive  cost  values.  This  is  done  in  the 
second  step.  The  data  are  divided  with  the  maximum  value  to  get  cost  ratios  in  the  third 
step.  The  exponential  function  is  now  used  to  derive  values  for  the  ratios.  Figure  3.15 
helps  to  understand  the  modification  process. 
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Figure  3.15  Finding  the  ratios  for  hardware 


Figure  3.16  shows  the  single  dimensional  value  function  for  Delta  O&S  parts. 


Figure  3.16  Delta  O&S  Parts 


3.6.6  Delta  O&S  Labor 

The  range  of  this  evaluation  measure  varied  from  $0  to  $5,000.  The  decision¬ 
maker  said  that  after  $600,  his  concerns  about  savings  were  increasing.  The  same  ratio 
technique,  explained  previously,  is  used  for  the  labor  costs.  Figure  3.17  shows  the  single 
dimensional  exponential  value  function  for  the  metric. 
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Figure  3.17  Delta  O&S  Labor 


3.6.7  Maintenance  Man  Hours 

Non-RTOC  factors  were  explained  previoulsy.  O&S  inspect  time  and  O&S  repair 
time  are  constructed  to  calculate  the  values.  However,  O&S  inspect  time  is  0  for  many 
alternatives.  Therefore,  as  discussed  earlier  maintenance  man  hours  serves  as  a  proxy  for 
these  two  evaluation  measures.  Maintenance  hours  are  the  sum  of  inspect  and  repair 
time.  The  costs  for  MMH  are  used  for  the  x-axis.  This  evaluation  measure  has  a  value 
from  $0  to  $600.  Fifty  dolars  is  used  as  the  changing  point  for  the  concerns  of  the 
decision-maker.  The  ratio  method,  explained  previously,  is  used  to  build  the  value 
function.  Figure  3.18  shows  the  exponential  function  for  the  metric. 
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Figure  3.18  Maintenance  man  hours 


3.7  Using  Additive  Value  Function 

The  additive  value  function  combines  the  single  dimensional  value  functions  that 
are  built  by  the  preferences’  of  the  decision-maker.  Equation  (Kirkwood,  1997:  243) 
explains  the  additive  value  functions. 

v(jc)  =  YjWi  *  Vi  (x) 

i  =  i 

V(x)  is  used  for  the  overall  value  of  alternative  x.  W;  is  the  global  weight  of  ith 
metric  and  Vi  (x)  is  defined  as  the  value  of  alternative  x  for  metric  i.  Thus,  overall  value 
is  equal  to  the  sum  of  multiplication  of  metrics’  global  weights  and  scores  of  alternatives 
for  each  metric.  The  most  important  property  for  an  additive  value  function  is  mutually 
preferential  independence.  The  property  was  checked  by  Kirkwood’s  protocol 
(Kirkwood,  1997:  App  7)  during  the  elicitation  of  metrics  phase  of  the  study.  The 
evaluation  measures  in  the  same  level  were  fixed  to  different  values  (beginning  values). 
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One  evaluation  measure  was  increased  or  decreased  while  the  others  were  held  constant. 


The  decision-maker  was  asked  if  it  was  possible  to  change  the  value  of  an  evaluation 
measure  while  the  others  were  fixed  and  if  he  would  change  his  preference  due  to  this 
change  of  the  evaluation  measure.  The  purpose  is  to  see  if  the  decision-maker  prefers  the 
highest  value  to  the  lowest  value  for  each  evaluation  measure.  The  decision-maker 
reported  that  he  would  not  change  his  preferences.  As  a  result,  the  analysis  assumed  that 
the  model  had  mutual  preferential  independence. 


3.8  Determining  the  Weights 

It  is  useful  to  review  some  properties  of  value  functions  to  understand  the 
procedure  for  determining  the  weights.  Zero  is  defined  as  the  least  preferred  level  and  1 
is  defined  as  the  most  preferred  level  for  corresponding  value  function  (Kirkwood, 
1997:68).  This  bounds  value  function  between  zero  and  1. 

The  weight  for  an  evaluation  measure  is  verbalized  as  “  the  value  increment  that 
is  received  from  moving  the  score  on  that  evaluation  measure  from  its  least  preferred 
value  to  its  most  preferred  value”  (Kirkwood,  1997:68).  The  concept  is  to  ask  how  much 
value  is  gained  from  lowest  to  highest  value  in  one  evaluation  measure  compared  to  one 
another.  The  procedure  is  done  for  evaluation  measures  on  the  same  level  to  get  rid  of 
the  problem  of  swinging  weights.  Swinging  weights  requires  the  decision-maker  to 
consider  the  ranges  of  all  the  evaluation  measures  in  different  levels.  The  procedure  used 
in  the  study  helped  the  decision-maker  to  provide  a  ratio  of  importance. 
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The  global  weights  of  the  each  evaluation  measure  are  used  in  the  additive  value 
function.  The  difference  between  local  and  global  weights  can  be  explained  in  a  simple 
example  as  shown  in  Figure  3.19. 


Figure  3.19  Local  and  Global  Weights 

The  global  weight  for  Measure  of  Merit  2  = 

=  Local  Weight  (Measure  Merit  2)*  Global  Weight  (Sub-Objective  A) 

=  0.40*0.75 

=  0.30 

The  calculations  for  weights  were  done  in  Logical  Decisions.  CEP,  kill  ratio  and 
survivability  weights  are  defined  arbitrarily  for  the  model  completeness.  This  does  not 
affect  the  results  of  the  study  because  engine  modifications  do  not  improve  the  F-16 
combat  capability  as  stated  earlier.  The  weights  are  shown  in  Figure  3.20. 
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Best  Eng.  Mods 
1.000 


Figure  3.20  Global  Weights  of  the  F-16  Capability  Enhancement  Model 


As  shown  in  the  previous  diagram,  the  evaluation  measure  UER  drivers  have  the  biggest 
impact  for  the  overall  value.  Safety  evaluation  measures  number  of  Class  A  accidents 


(Initial-Threshold/Final)  and  number  of  Class  A  accidents  (Mitigated-Threshold/Final) 
are  other  important  drivers  in  the  model.  However,  the  differences  among  the  global 
weights  of  the  evaluation  measures  are  not  significant.  This  property  makes  our  study  a 
true  mutidimensional  decision-making  problem.  It  is  not  possible  to  see  the  winning 
alternatives  without  applying  the  techniques  of  decision  analysis.  Figure  3.21  shows  the 
impact  of  the  evaluation  measures  on  the  overall  value  model.  This  figure  shows  the 
proportional  relative  importance  of  the  evaluation  measures  when  CEP,  kill  ratio  and 
survivability  are  not  included  in  the  model. 


Figure  3.21  Global  Weights  of  the  Evaluation  Measures 
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The  impact  of  the  some  evaluation  measures  like  UER  drivers  and  number  of  Class  A 
accidents  are  about  the  same.  This  research  is  expected  to  be  weight  sensitive  depending 
on  the  global  weights  of  the  evaluation  measures.  Sensitivity  analysis  of  the  research  will 
be  presented  in  Chapter  4. 

3.9  Modeling  Assumptions 

The  F-16  Capability  Enhancement  Model  is  a  new  step  toward  a  systematic 
approach  of  selecting  engine  modifications  for  the  aircraft.  There  is  no  background 
study.  There  are  separate  teams  working  on  the  different  aspects  of  the  problem.  This 
study  is  a  result  of  a  small  team  of  individuals.  The  goal  is  to  provide  a  framework  for 
comparing  modifications. 

The  hierarchy  is  built  as  a  full  model  to  help  future  researchers.  However,  the 
study  is  limited  to  the  engine  modifications  due  to  time  and  data  constraints.  Using 
engine  modifications  for  checking  the  model  accuracy  is  adequate  to  validate  the  model. 

It  is  assumed  that  engine  modifications  will  not  have  a  big  impact  on  the  combat 
score.  The  combat  score  sub-objective  is  weighted  zero  in  the  model.  Another  reason  for 
zero  weighting  the  combat  score  is  lack  of  data.  Data  availability  is  the  main  driver  in 
this  study. 

The  alternatives  used  in  the  study  are  assumed  to  represent  a  typical  sample  of  the 
F-16  modifications.  Ten  alternatives  are  selected  for  analyses  in  Chapter  4.  The  F-16 
SPO  identified  the  alternatives  to  test  the  model  robustness. 


75 


The  data  for  the  alternatives  are  assumed  to  be  independent,  applying 
modification  A  does  not  change  the  result  from  applying  modification  B.  The  data  for 
combat  capability,  safety,  and  operations  costs  are  also  assumed  to  be  independent  for  a 
specific  modification.  A  simple  example  helps  clarify  the  issue.  If  a  new  type  of  air-to- 
air  weapon  is  installed  on  the  F-16,  independence  assumes  that  the  vibration  created  by 
this  weapon  does  not  affect  engine  stalls  or  operations  costs  of  the  engine  for  future 
developments. 

Some  of  the  assumptions  in  this  study  should  be  relaxed  in  the  future  studies. 
Relaxing  the  assumptions  depends  on  the  available  data  rather  than  the  decision  analysis 
process. 

3.10  Methodology  Summary 

In  this  chapter,  reasons  for  selecting  the  VFT  approach  were  discussed.  Next,  the 
modeling  problems  and  solutions  were  introduced.  The  final  model  and  definitions  of  the 
values  were  explained.  Checking  the  model  independence  for  using  the  additive  value 
functions  was  explained.  Determining  the  weights  of  the  evaluation  measure  was 
discussed  with  an  example.  Finally  modeling  assumptions  were  highlighted  to  complete 
the  chapter. 
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Chapter  4:  Results  and  Analysis 


4.1  Introduction 

The  F-16  Capability  Enhancement  Model  is  divided  into  two  parts  to  help  explain 
the  analysis.  The  first  part  is  the  decision  weights  assigned  by  the  decision-maker.  The 
second  part  is  the  sub-model.  Figure  4.1  shows  these  parts. 
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The  analysis  was  performed  after  the  value  model  for  F-16  modification  process 
was  developed.  First,  the  F-16  Capability  Enhancement  model  is  applied.  This  allows 
ranking  of  modifications.  Three  different  rankings  are  performed  due  to  the  decision¬ 
maker’s  preferences  in  this  part.  The  first  ranking  is  based  on  the  importance  of  the 
mission  type  and  war  capabilities  of  the  fighter.  The  second  is  based  on  the  fleet  size  of 
the  aircraft.  The  last  ranking  is  for  the  Pratt  Whitney  Company  (PW),  who  is  one  of  the 
biggest  engine  providers  for  the  USAF.  Cost  analysis  is  performed  to  quantify  the 
benefit  per  dollar.  For  each  modification,  the  rankings  for  mission  type  and  fleet  size  are 
repeated  to  see  the  changes  due  to  costs. 

Second,  a  benefit/cost  ratio  greedy  algorithm  is  applied  to  get  insights  from  the 
benefit/cost  ratios.  In  this  algorithm,  we  simply  take  as  many  items  as  possible  without 
exceeding  our  budget  constraint,  beginning  with  the  biggest  benefit/cost  ratio.  The  result 
is  not  guaranteed  to  be  optimal.  Optimization  techniques  are  applied  to  maximize  the 
benefit  of  the  modifications  to  the  F-16  aircraft.  This  part  maximizes  the  value  of 
modifications  while  using  the  budget  as  a  constraint.  Integer  programming  is  used  to  find 
an  effective  portfolio.  Integer  programming  techniques  may  result  in  an  optimized 
portfolio  for  a  small  problem.  However,  an  optimized  portfolio  is  not  guaranteed  for  a 
large-scale  decision-making  model  with  many  alternatives  due  to  the  limitations  of 
integer  programming.  The  limitations  of  integer  programming  are  beyond  the  scope  of 
this  research.  A  detailed  discussion  about  integer  programming  and  its  limitations  can  be 
found  in  Winston’s  book  (Winston,  1994). 
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Finally,  what  if  and  sensitivity  analysis  are  performed  for  the  sub-model  to  see 
how  stable  the  value  model  is  with  changes  in  weights.  These  analyses  did  not  take  the 
decision  weights  (upper  tier  in  the  model)  into  consideration  since  they  are  defined  by  the 
preferences  of  the  F-16  Capability  Enhancement  Model  users. 

4.2  Model  Application  and  Ranking  the  Alternatives 

The  F-16  Capability  Enhancement  Model  is  used  to  analyze  the  benefit  to  the  F- 
16  aircraft  of  ten  modifications  currently  in  progress  or  planned  in  near  future.  However, 
there  are  12  alternatives  for  the  sub- model  when  the  modifications’  data  are  separated  for 
different  blocks.  The  data  section  of  Figure  3.5  explains  the  data  modification  process. 

4.2.1  The  F-16  Capability  Enhancement  Model  Ranking  for  Mission  Type  and  War 
Capabilities 

The  decision  weights  in  this  section  are  assigned  to  blocks  depending  on  their  war 
capabilities  in  this  part  of  the  study.  The  F-16  Block  40/42  and  50/52  aircraft  are 
valuable  for  the  USAF  in  wartime.  Aircraft  blocks  10/15/25/30/32  had  0  decision 
weights  since  these  aircraft  are  used  for  purposes  like  training,  testing,  and  etc.  Table 
4.1,  shown  below,  represents  the  decision  weights  of  the  aircraft  blocks. 


Table  4.1  Decision  Weights 


Aircraft 

Blocks 

10 

15 

25 

30 

32 

40 

42 

50 

52 

Total 

29991 

0 

0 

0 

0 

0 

0.1 

0.1 

0.4 

0.4 

1 
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Block  50/52  aircraft  are  heavily  weighted  because  of  their  unique  mission  capabilities  of 
suppression  of  enemy  air  defense  (SEAD).  These  weights  may  vary  due  to  the  changing 
requirements  of  the  decision-maker,  but  are  assumed  constant  for  this  analysis.  The 
weights  in  Table  4.1  are  used  to  show  the  efficiency  and  flexibility  of  the  model.  The 
alternatives  are  ranked  after  the  decision  weights  discussion.  Table  4.2  shows  the  final 
rankings  for  the  F-16  Capability  Enhancement  Model  as  defined  in  Figure  4.1. 


Table  4.2  Ranking  for  Mission  Type  and  War  Capabilities 


Engine  Types 

Alternatives 

Blocks 

Benefits 

GE  129 

Laser  Shock  Peen 

50 

0.1354 

GE  129 

Turbine  Frame  Outer  Liner 

50 

0.0767 

PW  229 

2nd  Fan  Stator 

52 

0.0753 

PW  229 

#4  Bearing  Seals 

52 

0.0746 

PW  229 

Oil  Filter  Housing 

52 

0.0744 

PW  229 

2nd  Turbine  Blade 

52 

0.0447 

PW  229 

Fuel  Nozzle  Damping 

52 

0.0331 

GE  100 

Ejector  Nozzle 

30-40 

0.0217 

GE  100 

DEC  Upgrade 

30-40 

0.0211 

PW  229 

Rear  Fan  Duct  Patch 

52 

0.0065 

The  GE 129  Laser  Shock  Peen  modification  provides  the  most  benefit  to  the  F-16 
aircraft  as  determined  by  the  model.  This  is  due  to  its  being  used  in  block  50  aircraft. 

The  status  quo  case  was  not  used  for  this  study,  since  all  the  modifications  analyzed  have 
a  potential  of  making  a  contribution  to  the  F-16  capabilities.  Instead,  hypothetical  best 
and  worst  cases  were  created  to  see  the  areas  for  development  of  the  alternatives. 
Hypothetical  worst  and  best  cases’  scores  for  evaluation  measures  are  defined  by  using 
the  best  and  worst  scores  of  all  alternatives  under  each  evaluation  measure.  Therefore,  it 
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is  not  expected  that  the  best  case  scores,  1,  and  worst  case  scores,  0,  for  all  objectives  due 
to  the  range  of  the  evaluation  measures.  The  evaluation  measure  ranges  were  defined 
depending  on  the  requirements  of  the  decision-maker  as  explained  in  Chapter  3.  Table 
4.3,  shown  below,  demonstrates  a  simple  example. 


Table  4.3  An  example  for  Hypothetical  Best  and  Worst  Cases 

Objective  1  Objective  2  Objective  3 

Global  Weight=0.33  Global  Weight=0.33  Global  Weight=0.33 


Score 

Value 

Score 

Value 

Score 

Value 

Total  Value 

Range 

0-10 

0-1 

0-100 

0-1 

0-1 

0-1 

Alternative  1 

8 

0.8 

50 

0.50 

0.4 

0.4 

0.56 

Alternative  2 

6 

0.6 

25 

0.25 

0.7 

0.7 

0.51 

Alternative  3 

2 

0.2 

85 

0.85 

0.1 

0.1 

0.38 

Best  Case 

8 

0.8 

85 

0.85 

0.7 

0.7 

0.78 

Worst  Case 

2 

0.2 

25 

0.25 

0.1 

0.1 

0.18 

The  decision-maker  defines  ranges  for  evaluation  measures  in  this  example. 
Single  dimensional  value  functions  are  assumed  to  be  piecewise  linear  functions.  The 
best  case  takes  the  scores  from  alternative  1  in  objective  1,  alternative  3  in  objective  2, 
and  alternative  2  in  objective  3.  This  results  in  a  total  value  of  0.78  assuming  equal 
global  weights  for  each  objective.  Similarly,  worst  case  takes  the  scores  from  alternative 
3  in  objective  1,  alternative  2  in  objective  2,  and  alternative  3  in  objective  3.  This  results 
in  a  total  value  of  0.18.  Thus,  this  example  shows  that  the  value  of  best  case  may  not  be 
equal  to  1  and  the  value  of  worst  case  may  not  be  equal  to  0. 

Figure  4.2,  shown  below,  represents  the  contribution  of  each  objective  for  the 
modifications  specific  to  the  block  type  in  the  sub-model.  The  ranking  in  Figure  4.2  does 
not  take  the  decision  weights  into  consideration  because  the  decision  weights  are  defined 
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as  the  changing  preferences  of  the  decision-maker.  The  goal  is  to  show  the 
modifications’  achievement  level  in  the  sub-model.  There  are  12  modifications  that  are 
scored  in  the  sub-model.  Figure  3.5  in  Chapter  3  explains  the  data  process. 


Ranking  for  Best  Eng.  Mods  Goal 


Alternative  Value 

Hypot.  Best  0.574 

129  LaserShockPeen  0.338 
100  Ejector  Nozzle40  0.215 
100  Dec  Upg.30  0.214 

100  Dec  Upg.40  0.214 

129  FrameOuterLiner  0.201 
229  2ndFanStator  0.188 
229  #4  BearingSeals  0.187 
229  OilFilterHousing  0.187 
100  Ejector  Nozzle30  0.134 
229  2nd  Turb.Blade  0.112 
229  FuelNozzleDamp  0.083 
229  R.FunDuctPatch  0.016 
Hypot.  Worst  0.011 


■  Combat  Cap.  □  Safety  □  Ops.  Costs 


Preference  Set  =  F-16  Capability  Enhancement  Model 

Figure  4.2  Fundamental  Objectives ’  Contributions 


The  hypothetical  best  case  gets  a  value  of  0.574.  The  closest  modification  to  hypothetical 
best  case  is  GE  129  Laser  Shock  Peen.  Improving  the  combat  capability  of  the  aircraft  is 
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the  most  important  ability  of  this  modification.  Contribution  to  safety  is  another  aspect 
of  the  modification  and  is  considered  significant  by  the  decision-maker.  However,  the 
operations  cost  of  the  modification  is  highly  expensive.  The  cost  must  be  improved 
(decreased)  for  better  results.  The  second  modification  in  the  ranking  is  GE 100  Ejector 
Nozzle  ( Block  40).  This  modification  is  a  safety  modification  and  it  does  not  have  any 
contribution  to  the  F-16  combat  capability.  The  other  GE  engine  modifications  score 
close  to  each  other.  The  only  exception  is  the  GE  100  Ejector  Nozzle  (Block  30).  The 
operations  cost  of  the  engines  depends  on  the  remaining  fleet  life.  The  remaining  fleet 
life  for  the  block  40  engines  are  longer  than  block  30  engines.  Thus,  the  GE  100  Ejector 
Nozzle  (Block  30)  modification  scores  weakly  when  it  is  compared  to  the  GE  100  Ejector 
Nozzle  (Block  40)  modification  due  to  its  high  operations  costs.  PW  229  2nd  Fan  Stator, 

# 4  Bearing  Seals  and  Oil  Filter  Housing  are  the  first  modifications  in  the  list  for  PW  229 
engine.  The  2nd  Fan  Stator  is  a  safety  modification  and  does  not  contribute  to  the  combat 
capability  objective.  The  last  two  modifications  for  PW  229  (Fuel  Nozzle  Damp  and  Rear 
Fun  Duct  Patch )  score  very  low  and  the  implementation  of  these  modifications  should 
not  be  considered  as  improving  the  aircraft  capabilities.  The  overall  result  is  that  the  GE 
engine  modifications  are  achieving  better  results  than  PW  engine  modifications  in  the 
sub-model. 

The  sub-model’s  combat  capability,  safety,  and  operational  costs’  individual 
rankings  can  also  be  shown.  The  Figure  4.3  shows  the  sub-model’s  ranking  for  the 
combat  capability  objective.  However,  the  ranking  was  done  only  for  aircraft  availability 
since  the  combat  score  was  weighted  0  in  the  model.  Note  that  the  decision  weights  (top 
tier)  are  not  used  in  the  ranking  for  combat  capability,  safety,  and  operations  costs,  since 


83 


they  depend  on  the  preferences  of  the  decision-maker.  The  goal  of  Figure  4.3  is  to  show 
the  weak  and  strong  properties  of  the  modifications  used  in  the  sub-model  and  provide 
better  insights  for  the  decision-maker. 


Ranking  for  Combat  Cap.  Goal 

Alternative 

Hypot.  Best 
129  LaserShockPeen 
229  OilFilterHousing 
229  #4  BearingSeals 
129  FrameOuterLiner 
229  2nd  Turb.Blade 
100  Dec  Upg.30 
100  Dec  Upg.40 
229  2ndFanStator 
229  FuelNozzleDamp 
229  R.FunDuctPatch 
100  Ejector  Nozzle30 
100  Ejector  Nozzle40 
Hypot.  Worst 


■  Combat  Score  □  A/C  Avail. 


Preference  Set  =  F-16  Capability  Enhancement  Model 

Figure  4.3  Ranking  for  Combat  Capability 


The  Logical  Decisions  software  was  used  for  this  part  of  the  study.  If  the  analyst  wants 
to  rank  the  alternatives  depending  on  an  objective,  the  software  sets  the  global  weight  of 
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that  objective  to  1.  However,  the  weight  ratios  defined  by  the  decision-maker  stay  the 
same.  The  global  weights  were  adjusted  to  0.7  and  0.3  for  combat  capability  and  aircraft 
availability  in  this  section.  The  global  weight  for  combat  capability  is  set  to  0  as 
discussed  earlier.  Therefore,  the  only  criterion  in  this  portion  of  the  model  is  availability. 
The  HER  driver  evaluation  measure  score  (only  evaluation  measure  for  sub-objective 
A/C  availability)  for  hypothetical  best  case  is  the  same  score  as  GE 129  Laser  Shock 
Peen  modification.  The  PW  229  Oil  Filter  Housing  and  #4  Bearing  Seals'  values  are 
very  close  to  the  hypothetical  best  case.  It  can  be  stated  that  there  is  minimal  difference 
between  the  impacts  of  GE  129  Laser  Shock  Peen,  PW  229  Oil  Filter  Housing  and  # 4 
Bearing  Seals  on  the  aircraft  availability.  However,  the  values  following  these 
modifications  decrease  dramatically.  The  other  important  insight  is  that  if  aircraft 
availability  is  an  important  issue,  the  ranking  for  GE  100  DEC  Upgrade  { Block  30/40) 
changes  severely  due  to  its  decreasing  value  in  the  sub-model.  The  last  5  modifications 
on  the  list  (PW  229  2nd  Fan  Stator,  PW  229  Fuel  Nozzle  Dump,  PW 229  Rear  Fun  Duct 
Pacth,  and  GE  100  Ejector  Nozzle  Block  30/Block  40)  do  not  impact  aircraft  availability 
at  all.  The  decision-maker  has  to  keep  in  mind  that  combat  capability  is  the  most 
important  part  of  this  model,  considering  its  global  weight.  Improving  the  performances 
of  the  modifications  for  aircraft  availability  changes  the  overall  values  significantly. 

The  total  change  in  class  A  mishap  rate  and  installation  schedule  are  the  sub¬ 
objectives  of  the  safety.  The  sub-model’s  ranking  for  Safety  is  shown  in  Figure  4.4 
below.  Safety  is  an  important  concern  in  the  study  since  the  human  life  cannot  be 
replaced.  The  decision  weights  are  not  considered  in  the  ranking. 
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Ranking  for  Safety  Goal 


Alternative  Value 

Hypot.  Best  0.909 

129  LaserShockPeen  0.675 
229  2ndFanStator  0.606 
100  Dec  Upg.30  0.518 
100  Dec  Upg.40  0.518 
129  FrameOuterLiner  0.478 
229  FuelNozzleDamp  0.274 
229  #4  BearingSeals  0.199 
229  OilFilterHousing  0.196 
229  2nd  Turb.Blade  0.184 
100  Ejector  Nozzle30  0.151 
100  Ejector  Nozzle40  0. 150 
229  R.FunDuctPatch  0.050 
Hypot.  Worst  0.035 


■  Total  Change  □  Install.  Schedule 


Preference  Set  =  F-16  Capability  Enhancement  Model 

Figure  4.4  Ranking  for  Safety 


The  hypothetical  best  case  score  is  0.909  for  the  safety  objective.  The  GE 129  Laser 
Shock  Peen  has  the  closest  value  to  the  hypothetical  best  case.  This  modification  scores 
high  under  total  change  in  class  A  mishap  rates  sub-objectives.  However,  the  installation 
schedule  of  this  modification  has  to  be  improved  for  better  results.  The  PW  229  2nd  Fan 
Stator  is  the  second  modification  on  the  list  (excluding  the  hypothetical  best  case).  The 
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installation  schedule  score  for  this  modification  is  the  best  one  under  installation  schedule 


sub-objective  (WAPM  value).  Increasing  the  safety  capability  of  this  modification  will 
increase  modification  score.  However,  this  might  not  be  possible  for  technical  reasons. 
The  GE 100  DEC  Upgrades  (Block  30/40)  are  next  on  the  list.  The  installation  schedule 
for  these  modifications  must  be  improved  to  increase  the  values  of  these  modifications. 
The  GE  129  Frame  Outer  Liner  and  PW  229  Fuel  Nozzle  Damp  are  next  in  the  ranking. 
The  last  5  modifications  on  the  list  (PW  229  # 4  Bearing  Seals,  PW  229  Oil  Filter 
Housing,  PW  229  2nd  Turbine  Blade,  GE  100  Ejector  Nozzle  Block  30/Block  40)  do  not 
contribute  to  the  objective  of  reducing  class  A  mishap  rates  for  the  F-16.  They  score 
average  under  the  installation  schedule  sub-objective.  The  PW  229  Rear  Fun  Duct  Patch 
is  the  final  modification  on  the  ranking.  If  class  A  mishap  rates  become  a  bigger  concern, 
these  last  6  modification  do  not  have  to  be  considered  as  valuable  modifications. 
Improving  the  installation  schedule  for  GE  129  Laser  Shock  Peen,  and  GE  100  DEC 
Upgrades  ( Block  30/40)  increase  the  values  of  these  modifications  considerably. 

Operations  cost  is  divided  into  two  sub-objectives,  RTOC  factors  and  non-RTOC 
factors.  Figure  4.5,  shown  below,  shows  the  sub-model’s  ranking  for  operational  costs. 
The  decision  weights  are  not  used  in  the  ranking  as  explained  previously. 

Here  the  hypothetical  best  case  gets  a  value  of  0.852.  The  hypothetical  best  case 
gets  its  score  mostly  from  GE  100  Ejector  Nozzle  (Block  40)  modifications.  This 
modification  is  considered  the  first  one  to  be  implemented  if  the  operations  costs  were  the 
most  important  issue  in  the  problem. 
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Ranking  for  Ops.  Costs  Goal 


Alternative  Value 

Hypot.  Best  0.852 

100  Ejector  Nozzle40  0.849 

100  Ejector  Nozzle30  0.443 

100  Dec  Upg.30  0.101 

100  Dec  Upg.40  0.101 

229  2nd  Turb.B lade  0.050 

229  2ndFanStator  0.031 

129  LaserS  hoc  kPeen  0.021 

229  OilFilterHousing  0.007 

229  R.FunDuctPatch  0.006 

129  FrameOuterLiner  0.004 

229  #4  BearingSeals  0.004 

229  FuelNozzleDamp  0.004 

Hypot.  Worst  0.001 


n 

D 

I 


RTOC 


I  I  non-RTOC 


Preference  Set  =  F-16  Capability  Enhancement  Model 

Figure  4.5  Ranking  for  Operations  Cost 


The  GE 100  Ejector  Nozzle  ( Block  30)  is  the  2nd  modification  in  the  list.  The  blocks 
score  differently  depending  on  their  costs  data  and  this  data  depend  on  the  remaining  fleet 
life  as  explained  previously  (RTOC  and  non-RTOC  values).  The  remaining 

J 

modifications  on  the  lists  score  very  low.  The  GE  129  Laser  Shock  Peen  and  PW  229  2 
Fan  Stator  are  important  modifications  for  combat  capability  and  safety.  However,  they 
score  disappointingly  under  operations  costs.  These  low  scoring  under  operations  costs 
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may  impair  their  overall  contributions.  Improving  their  cost  scores  may  help  these 
modifications  to  become  clear  winners  for  sub-model.  The  last  5  modifications  do  not 
have  to  be  considered  if  the  costs  factors  are  driving  the  decision-maker.  After  reviewing 
these  figures,  the  most  important  insights  for  the  decision-maker  are: 

•  The  installation  schedule  scores  for  GE  100/129  engines  must  be  improved. 

•  The  operations  costs  of  PW  229  and  GE  129  engine  modifications  must  be 
decreased.  If  the  operations  cost  is  a  real  concern,  GE  100  modifications  have 
to  be  considered  as  the  most  valuable  ones. 

Figures  4.3,  4.4,  and  4.5  present  a  big  picture  of  the  achievements  of  alternatives  for  the 
objectives  (1st  tier).  The  contribution  for  each  evaluation  measure  in  the  sub-model  can 
also  be  shown  to  see  how  well  the  modifications  achieve  these  metrics.  Figure  4.6  shows 
the  contributions  for  each  evaluation  measure  for  F-16  Capability  Enhancement  Model. 
There  are  10  evaluations  measures  in  the  model.  Kill  ratio,  CEP,  and  survivability 
evaluation  measures  did  not  have  any  contribution  to  the  overall  values  since  they  are 
weighted  0  in  the  model.  Figure  4.6,  shown  below,  is  a  more  detailed  look  for  all  the 
modifications  used  in  this  study. 

The  GE  129  Laser  Shock  Peen  is  the  most  effective  modification  in  the  list.  It 
was  explained  before  the  operations  costs  and  installation  schedule  of  this  alternative 
must  be  improved  for  better  results.  Figure  4.6  shows  areas  for  improvement  for  each 
modification.  Improving  the  scores  for  O&S  parts,  WAPM,  or  the  MMH  cost  for  GE  129 
Laser  Shock  Peen  improve  its  overall  value  dramatically  depending  on  the  global  weight 
of  the  evaluation  measures. 


89 


Ranking  for  Best  Eng.  Mods  Goal 


Alternative  Value 


Hypot.  Best 
129  LaserShockPeen 
100  Ejector  Nozzle40 
100  Dec  Upg.30 
100  Dec  Upg.40 
129  FrameOuterLiner 
229  2ndFanStator 
229  #4  BearingSeals 
229  OilFilterHousing 
100  Ejector  Nozzle30 
229  2nd  Turb.Blade 
229  FuelNozzleDamp 
229  R.FunDuctPatch 
Hypot.  Worst 


0.016  I 
0.011  I 


■  UER  Drivers  □  Class  A's(I-T/F)  □  Class  A’s(M-T/F) 

■  O&S  Parts  ■  MMH  ■  Wt' d  Annual  Percentage 

■  O&S  Labor  m  Kill  Ratio  m  CEP 

HI  Survivability 


Preference  Set  =  F-16  Capability  Enhancement  Model 

Figure  4.6 Evaluation  Measures’  Contributions 


The  GE 100  Ejector  Nozzle  ( Block  40)  is  the  second  modification  on  the  list. 
However,  the  decision-maker  has  to  be  careful  at  this  point.  The  overall  value  for  this 
modification  is  mostly  originated  from  operations  costs.  Considering  this  result,  this 
modification  may  not  be  an  effective  option  for  F-16  Capability  Enhancement.  The  GE 
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100  DEC  Upgrade  ( Block  30/40)  and  GE  129  Frame  Outer  Liner  values  are  very  close  to 
each  other.  O&S  parts,  WAPM,  or  the  cost  for  MMH  are  the  areas  that  may  be  enhanced 
for  these  modifications.  However,  the  GE  129  Frame  Outer  Liner's  score  for  O&S 
Labor  can  improve  the  modification  ranking.  The  first  PW  229  modification  in  the 
ranking  is  the  PW  229  2nd  Fan  Stator.  The  values  for  PW  229  2nd  Fan  Stator,  PW  229  #4 
Bearing  Seals,  and  PW  229  are  very  close.  Therefore,  improving  scores  for  different 
areas  (score  for  O&S  Labor,  O&S  parts,  or  the  cost  for  MMH)  may  change  the  ranking 
noticeably.  The  next  modification  is  the  GE  100  Ejector  Nozzle  (Block  30)  which  scores 
extremely  low  when  compared  to  GE  100  Ejector  Nozzle  (Block  40).  The  remaining 
modifications  are  not  valuable  for  the  F-16  aircraft  based  on  the  values  of  the  decision¬ 
maker. 

4.2.2  The  F-16  Capability  Enhancement  Model  Ranking  for  Fleet  Size 

Normalization  for  fleet  size  is  used  to  find  decision  weights.  The  total  aircraft 
number  is  used  for  the  normalization  value.  The  number  of  specific  block  aircraft  is 
divided  by  the  total  number  of  aircraft  to  get  the  normalization  figures.  Table  4.4 
summarizes  the  process  to  find  the  normalized  values. 
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Table  4.4  Normalization  for  Aircraft  Number 


Blk  Total 

Normalized 

Values 

10A 

15 

35 

35/1436 

10B 

20 

15A 

88 

109 

109/1436 

15B 

21 

0.076 

25C 

212 

■sum 

25D 

32 

IHHDZS 

324 

366 

■EUSESE 

42 

32C 

49 

53 

53/1436 

32  D 

4 

40C 

213 

240 

■EEKSE 

40  D 

27 

42C 

140 

186 

186/1436 

42  D 

46 

0.130 

50C 

154 

181 

181/1436 

50D 

27 

52C 

42 

54 

52  D 

12 

Total 

HESS 

Table  4.5  shows  the  decision  weights  used  in  this  section  of  the  analysis. 


Table  4.5  Decision  Weights  for  Fleet  Size 


Aircraft 

Blocks 

B9 

m 

30 

32 

40 

42 

50 

H 

Total 

0.024 

0.255 

0.037 

0.167 

EXES 

1 

Table  4.6  shows  the  F-16  Capability  Enhancement  Model’s  ranking.  The  Digital  engine 
component  (DEC)  upgrade  for  GE 100  engine  is  the  winning  alternative.  This  was 
expected,  since  the  block  30  aircraft  comprise  the  largest  portion  of  the  USAF  fleet. 
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Table  4.6  Ranking  for  Fleet  Size 


Engine  Types 

Alternatives 

Blocks 

Benefits 

GE  100 

DEC  Upgrade 

30-40 

0.08912 

GE  100 

Ejector  Nozzle 

30-40 

0.07061 

GE  129 

Laser  Shock  Peen 

50 

0.04266 

GE  129 

Turbine  Frame  Outer  Liner 

50 

0.02418 

PW  229 

2nd  Fan  Stator 

52 

0.00708 

PW  229 

#4  Bearing  Seals 

52 

0.00701 

PW  229 

Oil  Filter  Housing 

52 

0.00700 

PW  229 

2nd  Turbine  Blade 

52 

0.00420 

PW  229 

Fuel  Nozzle  Damping 

52 

0.00312 

PW  229 

Rear  Fan  Duct  Patch 

52 

0.00061 

The  GE  100/129  engine  modifications  receive  higher  values  than  PW  229  engine 
modifications.  This  is  predictable  since  they  form  the  largest  percentage  of  USAF  fleet. 
The  most  important  PW  229  modification  appears  to  be  the  PW  229  2nd  Fan  Stator. 
However,  PW  229  # 4  Bearing  Seals  and  PW 229  Oil  Filter  Housing’s  values  are  close  to 
the  value  of  PW  229  2nd  Fan  Stator.  The  subsequent  modifications  on  the  list  do  not  have 
big  impacts  on  the  enhancement  of  F-16  capabilities  when  normalized  by  fleet  size. 

The  graphs  for  sub-model’s  combat  capability,  safety,  and  operational  cost 
contributions  are  the  same  with  mission  type  and  war  capabilities’  graphs  since  the  only 
changing  data  are  the  decision  weights  for  F-16  Capability  Enhancement  Model.  The 
decision  weights  do  not  impact  the  sub-model  performance  of  the  modifications.  Sub¬ 
model  is  based  on  the  data,  while  the  top  tier  changes  the  overall  rankings  due  to  the 
decision-maker’s  preferences.  As  seen  in  Table  4.5,  the  rankings  can  change 
dramatically. 
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4.2.3  The  F-16  Capability  Enhancement  Model  Ranking  for  Pratt  Whitney  Engines 


Many  decision-maker  questions  can  be  answered  because  of  the  flexibility  of  the 
decision  weights  in  the  F-16  Capability  Enhancement  Model.  One  important  question 
was  ranking  the  alternatives  for  different  manufacturers.  Table  4.7  shows  the  F-16 
Capability  Enhancement  Model  ranking  for  PW  engines.  Block  10/15/25/32/42/52 
aircraft  use  the  PW  engines.  Thus,  the  decision  weights  were  equally  distributed.  Each 
block  type  had  a  decision  weight  of  0.166  (The  total  decision  weight  =  1,  PW  blocks  = 
6).  This  simple  case  is  used  to  show  the  broad  capabilities  of  the  model.  However, 
different  decision  weights  may  be  built  depending  on  the  preferences  of  the  decision¬ 
maker.  Fleet  size  of  PW  engines  or  the  war  capabilities  of  different  PW  engines  may 
result  in  different  decision  weights  sets.  However,  this  simple  case  demonstrates  the 
flexibility  and  tries  to  keep  away  of  the  interaction  issues  that  may  be  a  result  of  changing 
preferences  of  the  decision-maker. 


Table  4.7  Ranking  for  PW  Engines 


Engine  Types 

Alternatives 

Blocks  1 

Benefits 

PW  229 

2nd  Fan  Stator 

52 

0.03136 

PW  229 

#4  Bearing  Seals 

52 

0.03108 

PW  229 

Oil  Filter  Housing 

52 

0.03101 

PW  229 

2nd  Turbine  Blade 

52 

0.01860 

PW  229 

Fuel  Nozzle  Damping 

52 

0.01381 

PW  229 

Rear  Fan  Duct  Patch 

52 

0.00271 

The  2nd  Fan  Stator  for  PW  229  is  the  1st  alternative.  However,  the  values  for  PW  229  #4 


Bearing  Seals  and  PW  229  Oil  Filter  Housing  are  very  close.  Therefore,  it  can  be  stated 
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that  there  is  minimal  difference  among  the  impacts  of  these  3  alternatives.  The  final 
modification,  PW 229  Rear  Fun  Duct  Patch  does  not  contribute  significantly  to  the  F-16 
capabilities.  The  graphs  for  sub-model’s  combat  capability,  safety,  and  operational  cost 
contributions  are  the  same  with  mission  type  and  war  capabilities’  graphs  as  explained 
previously. 

4.3  The  Cost  of  Enhancing  the  F-16  Capabilities 

It  is  not  completely  true  to  conclude  that  the  F-16  organizations  should  only  fund 
the  alternatives  resulting  in  the  greatest  benefit  to  the  F-16.  The  benefit  comes  at  a  price. 
A  value  analysis  approach  is  useful  for  cost  analysis  because  it  provides  a  systematic 
method  that  allows  everyone  involved  in  the  capital  budgeting  process  to  provide 
information  in  a  clear  and  mutually  understood  way  (Kirkwood,  1997:200-206).  Benefit/ 
cost  ratio  analysis  is  used  for  this  part  of  study.  It  is  a  widely  used  method  for  analyzing 
resource  allocation  decisions.  The  best  modifications  are  defined  as  the  ones  providing 
big  change  in  benefits  at  the  smallest  possible  cost.  The  acquisition  cost  in  this  part  of 
analysis  is  defined  as  the  cost  of  a  modification  for  a  single  engine.  Table  4.8  and  Table 
4.9  show  the  re-ranked  modifications  for  the  F-16  Capability  Enhancement  Model  from 
highest  to  lowest  benefit/cost  ratio  depending  on  the  decision  weights  for  Mission  Type 
and  War  Capability  and  Fleet  Size. 
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Table  4.8  Benefit/Cost  Ratio  Ranking  for  Mission  Type  and  War  Capability 


Engine  Types 

Alternatives 

Blocks 

Benefit 

Benefit/Cost 

GE  129 

Turbine  Frame  Outer  Liner 

50 

0.0767 

0.00029854 

PW  229 

#4  Bearing  Seals 

52 

0.0746 

0.00011782 

PW  229 

Oil  Filter  Housing 

52 

0.0744 

0.00002961 

PW  229 

Rear  Fan  Duct  Patch 

52 

0.0065 

0.00001836 

GE  129 

Laser  Shock  Peen 

50 

0.1354 

0.00000952 

PW  229 

Fuel  Nozzle  Damping 

52 

0.0331 

0.00000390 

PW  229 

2nd  Fan  Stator 

52 

0.0753 

0.00000384 

PW  229 

2nd  Turbine  Blade 

52 

0.0447 

0.00000112 

GE  100 

Ejector  Nozzle 

30-40 

0.0217 

0.00000017 

GE  100 

DEC  Upgrade 

30-40 

0.021 1 

0.00000012 

The  Turbine  Frame  Outer  Liner  for  GE  129  engine  is  the  winning  modification.  The  PW 
229  # 4  Bearing  Seals  and  PW  229  Oil  Filter  Housing  are  the  next  modifications  on  the 
list.  The  GE  129  Laser  Shock  Peen  is  fifth  due  to  its  expensive  acquisition  cost.  This 
position  is  a  striking  change  when  compared  to  Table  4.2.  The  2nd  Fan  Stator  for  PW 
229,  Digital  engine  component  (DEC)  upgrade  for  GE  100  are  other  modifications 
impaired  by  their  high  acquisition  expenses.  As  in  Table  4.7,  the  ranking  changes 
considerably  when  the  acquisition  costs  are  taken  into  consideration. 


Table  4.9  Benefit/Cost  Ratio  Ranking  for  Fleet  Size 


Engine  Types 

Alternatives 

Blocks 

Benefits 

Benefit/Cost 

GE  129 

Turbine  Frame  Outer  Liner 

50 

0.02418 

0.00009407 

PW  229 

#4  Bearing  Seals 

52 

0.00701 

0.00001108 

GE  100 

Ejector  Nozzle 

30-40 

0.07061 

0.00000543 

GE  100 

DEC  Upgrade 

30-40 

0.08912 

0.00000522 

GE  129 

Laser  Shock  Peen 

50 

0.04266 

0.00000300 

PW  229 

Oil  Filter  Housing 

52 

0.00700 

0.00000278 

PW  229 

Rear  Fan  Duct  Patch 

52 

0.00061 

0.00000173 

PW  229 

Fuel  Nozzle  Damping 

52 

0.00312 

0.00000037 

PW  229 

2nd  Fan  Stator 

52 

0.00708 

0.00000036 

PW  229 

2nd  Turbine  Blade 

52 

0.00420 

0.0000001 1 
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The  Turbine  Frame  Outer  Liner  for  GE  129  engine  is  the  winning  modification  for  this 
ranking  as  well.  The  benefit/cost  ratios  decrease  significantly  in  the  table.  The  PW 229 
# 4  Bearing  Seals  is  the  next  essential  modification  on  the  list.  The  GE  100  DEC 
Upgrade  does  not  have  an  important  ranking  position  owing  to  its  high  acquisition 
expenses  when  it  is  compared  to  Table  4.6.  Table  4.10,  shown  below,  represents  the 
benefit/cost  ratio  ranking  for  PW  engines. 


Table  4.10  Benefit/Cost  Ratio  Ranking  for  PW  Engines 


Engine  Types 

Alternatives 

Blocks 

Benefits 

Benefit/Cost 

PW  229 

#4  Bearing  Seals 

52 

0.03108 

0.0000491 

PW  229 

Oil  Filter  Housing 

52 

0.03101 

0.0000123 

PW229 

Rear  Fan  Duct  Patch 

52 

0.00271 

0.0000076 

PW  229 

Fuel  Nozzle  Damping 

52 

0.01381 

0.0000016 

PW  229 

2nd  Fan  Stator 

52 

0.03136 

0.0000016 

PW  229 

2nd  Turbine  Blade 

52 

0.01860 

0.0000005 

The  #4  Bearing  Seals  for  PW  229  is  highest  alternative  per  dollar  invested.  The  PW  229 
Oil  Filter  Housing  ranks  in  the  second  position.  The  first  striking  change  in  the  rankings 
is  the  new  place  of  PW  229  Rear  Duct  Fan  Patch.  This  modification  ranks  higher  due  to 
its  lower  acquisition  expenses.  The  other  important  ranking  change  is  the  position  of  PW 
229  2nd  Fan  Stator  when  acquisition  cost  is  important.  The  modification  suffers  due  to  its 
high  acquisition  costs. 

Benefit/cost  ratio  greedy  algorithm  is  applied  to  find  a  set  or  portfolio  of  best 
modifications.  In  this  step,  it  is  assumed  that  the  value  and  cost  of  a  modification  does 
not  change  when  it  was  combined  with  other  modifications.  Therefore,  it  is  possible  to 
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sum  the  benefits  and  costs  to  build  a  set  of  best  alternatives  (Kirkwood,  1997:203).  Table 
4.11  shows  the  results  for  Mission  Type  and  War  Capabilities.  The  total  cost  for  the 
modification  (acquisition  cost  for  an  engine  times  the  number  of  aircraft  modified  by  a 
specific  modification)  is  used  to  give  better  insights  about  the  most  effective  budgeting 
policy  for  the  F-16  organizations.  The  costs  are  expressed  in  thousands  of  dollars. 


Table  4.11  Greedy  Algorithm  Results  for  Mission  Type  and  War  Capabilities 


Engine  Costs 

Types  Alternatives  Blocks  Benefit  (thousand)  Benefit/Costs  CUM  Ben.  CUM  costs 


PW  229  #4  Bearing  Seals 

52 

0.0746 

48.7 

0.00153143 

0.07458 

48.7 

r^r-  mn  Turbine  Frame  Outer 
GE  129  Liner 

50 

0.0767 

77.9 

0.00098493 

0.15131 

126.6 

PW  229  Oil  Filter  Housing 

52 

0.0744 

193.5 

0.00038460 

0.22573 

320.1 

PW  229  Rear  Fan  Duct  Patch 

52 

0.0065 

27.3 

0.00023807 

0.23223 

347.4 

PW  229  Fuel  Nozzle  Damping 

52 

0.0331 

654.5 

0.00005064 

0.26537 

1001.9 

PW  229  2nd  Fan  Stator 

52 

0.0753 

1509.2 

0.00004987 

0.34063 

2511.1 

GE  129  Laser  Shock  Peen 

50 

0.1354 

4310.2 

0.00003141 

0.47602 

6821.3 

PW  229  2nd  Turbine  Blade 

52 

0.0447 

3064.7 

0.00001457 

0.52067 

9886.0 

GE  100  Ejector  Nozzle 

30-40 

0.0217 

103740.0 

0.00000021 

0.54237 

113626.0 

GE  100  DEC  Upgrade 

30-40 

0.021 1 

136218.6 

0.00000016 

0.56350 

249844.6 

The  PW  229  #4  Bearing  Seals  is  the  first  on  the  list.  However,  the  decision¬ 
maker  must  be  careful  while  considering  the  ranking  in  the  list.  The  rankings  mainly 
depend  on  the  total  number  of  aircraft  modified.  Therefore,  this  analysis  tries  to 
enlighten  the  budgeting  profile  question  of  the  ASC  without  providing  an  optimal 
solution.  The  GE 129  Turbine  Frame  Outer  Liner  is  the  2nd  ranked  modification.  The 
modifications  GE  100  Ejector  Nozzle  (Block30/40),  GE  129  Laser  Shock  Peen,  and  PW 
229  2nd  Fan  Stator  are  not  achieving  successful  results  because  of  their  total  costs.  Graph 
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shown  in  Figure  4.7  explains  the  relationship  between  cumulative  cost  and  cumulative 
benefit.  The  costs  are  expressed  in  thousands  of  dollars. 


Benefit/Cost  Analysis 


Figure  4.7  Plot  of  Cumulative  cost  versus  Cumulative  Benefit  for  Mission  Type  and 

War  Capabilities 


This  graph  shows  that  the  first  hundred  thousand  dollars  buy  substantial  benefits.  There 
is  not  much  additional  benefit  for  the  last  few  modifications  considering  their  excessive 
costs.  For  the  first  8  modifications,  the  graph  has  an  increasing  steep  line  showing  that 
cumulative  benefit  increase  dramatically  for  these  modifications.  ASC  may  get  a  total 
value  of  0.5  while  spending  less  than  10  million  dollars.  The  last  two  GE  100  engine 
modifications  propose  a  very  low  value  increment  for  a  big  price  change.  These 
modifications  may  be  left  out  of  an  effective  modification  set  without  impacting  the  very 
high  value  for  the  objective  function. 

Table  4.12,  shown  below,  demonstrates  the  results  for  Fleet  Size.  The  costs  are 
expressed  in  thousands  of  dollars. 
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Table  4.12  Greedy  Algorithm  Results  for  Fleet  Size 


Engine  Costs 

Types  Alternatives  Blocks  Benefits  (thousand)  Benefit/Costs  CUM  Ben  CUM  costs 


GE  129 

Turbine  Frame  Outer 
Liner 

50 

0.02418 

PW  229 

#4  Bearing  Seals 

52 

0.00701 

PW  229 

Oil  Filter  Housing 

52 

0.00700 

PW  229 

Rear  Fan  Duct  Patch 

52 

0.00061 

GE  129 

Laser  Shock  Peen 

50 

0.04266 

PW  229 

Fuel  Nozzle  Damping 

52 

0.00312 

PW  229 

2nd  Fan  Stator 

52 

0.00708 

PW  229 

2nd  Turbine  Blade 

52 

0.00420 

GE  100 

Ejector  Nozzle 

30-40 

0.07061 

GE  100 

DEC  Upgrade 

30-40 

0.08912 

77.9 

0.00031036 

0.02418 

77.9 

48.7 

0.00014397 

0.03119 

126.6 

193.5 

0.00003616 

0.03819 

320.1 

27.3 

0.00002238 

0.03880 

347.4 

4310.2 

0.00000990 

0.08146 

4657.6 

654.5 

0.00000476 

0.08458 

5312.1 

1509.2 

0.00000469 

0.09165 

6821.3 

3064.7 

0.00000137 

0.09585 

9886.0 

103740.0 

0.00000068 

0.16646 

113626.0 

136218.6 

0.00000065 

0.25558 

249844.6 

The  GE  129  Turbine  Frame  Outer  Liner  is  ranked  first.  GE  100  engine 
modifications  are  the  final  2  modifications  in  the  list  due  to  their  total  costs.  The  GE  129 
Laser  Shock  Peen  and  PW  229  2nd  Fan  Stator  are  valuable  modifications,  however,  their 
values  are  impaired  by  the  total  costs.  It  is  important  to  understand  that  this  ranking  is 
not  the  only  consideration  to  pick  the  most  beneficiary  modifications.  Table  4.12 
provides  insight  about  an  effective  modification  set.  Figure  4.8  shows  the  relationships 
between  cumulative  cost  and  cumulative  benefit.  The  costs  are  expressed  in  thousands  of 
dollars. 


Figure  4.8  Plot  of  Cumulative  Cost  versus  Cumulative  Benefit  for  Fleet  Size 
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The  graph  shows  that  the  dollars  invested  in  the  project  buy  substantial  benefit  for  the 
aircraft.  Decrease  in  the  value  shows  that  changing  decision  weights  for  fleet  size  results 
in  poor  outcomes.  However,  if  the  budget  increases,  the  cumulative  benefits  increase  in 
larger  amount  due  to  the  higher  slope  of  the  line  (compared  to  Figure  4.8).  The  decision¬ 
maker  would  have  to  decide  on  the  last  2  modifications  depending  on  the  budget  of  the 
organization. 

The  same  analysis  is  used  for  PW  engines.  Table  4.13  summarizes  the  process. 
The  costs  are  expressed  in  thousands  of  dollars. 


Table  4.13  Greedy  Algorithm  Results  for  PW  Engines 


Engine  Costs 

Types  Alternatives  Blocks  Benefits  (thousand)  Benefit/Costs  CUM  Ben  CUM  costs 


PW  229 

#4  Bearing  Seals 

52 

0.03108 

48.7 

0.00063809 

0.03108 

48.7 

PW229 

Oil  Filter  Housing 

52 

0.03101 

193.5 

0.00016025 

0.06208 

242.2 

PW229 

Rear  Fan  Duct  Patch 

52 

0.00271 

27.3 

0.00009919 

0.06479 

269.5 

PW  229 

Fuel  Nozzle  Damping 

52 

0.01381 

654.5 

0.00002110 

0.07860 

924 

PW  229 

2nd  Fan  Stator 

52 

0.03136 

1509.2 

0.00002078 

0.10996 

2433.2 

PW  229 

2nd  Turbine  Blade 

52 

0.01860 

3064.7 

0.00000607 

0.12856 

5497.9 

The  PW  229  #4  Bearing  Seals  is  the  first  ranked  modification  due  to  its  low  total  cost. 
Figure  4.9  explains  the  relationships  between  the  cumulative  benefits  and  the 


costs  for  modifying  PW  engines.  The  costs  are  expressed  in  thousands  of  dollars. 
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Figure  4.9  Plot  of  Cumulative  Cost  versus  Cumulative  Benefit  for  PW  Engines 


However,  a  more  effective  set  may  be  formed  by  not  applying  PW  229  Rear  Fun  Duct 
Patch.  This  modification  gives  a  very  little  benefit  improvement  for  the  aircraft.  All  PW 
engines  may  be  modified  for  less  than  5.5  million  dollars  due  to  the  limited  alternatives 
given  for  this  research.  This  highlights  the  relatively  inexpensive  nature  of  PW  engine 
modifications  when  compared  to  GE  engine  modifications. 

Despite  the  fact  that  benefit/cost  analysis  is  an  applicable  technique  for  portfolio 
selection  models,  the  process  has  some  limitations.  Kirkwood  explains  these  limitations 
in  his  book  (Kirkwood,  1997).  There  are  two  important  limitations  that  should  be 
understood  by  the  decision-maker.  First,  the  selected  portfolio  is  not  necessarily  optimal 
(Kirkwood,  1996:  200-205).  Second,  the  method  only  uses  a  single  budget  constraint 
(cost)  (Kirkwood,  1997:  206).  Linear  and  integer  programming  techniques  help  with 
portfolio  optimization  in  small  decision  problems.  However,  for  big  decision  models 
with  many  alternatives,  optimized  portfolios  are  not  guaranteed  by  linear  or  integer 
programming  techniques. 
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4.4  Optimizing  the  Portfolio 


For  this  part  of  the  analysis,  All-or-Nothing  or  Partial-Funding  policies  can  be 
used.  All-or-nothing  policies  are  the  case  where  a  modification  is  either  fully  funded  or 
dropped  from  the  desired  portfolio.  Partial-funding  allows  a  fraction  of  a  modification  to 
be  funded.  This  study  uses  all  or  nothing  funding  policy.  The  following  binary,  or  0-1, 
math  is  used  to  determine  the  optimal  portfolio  solution  for  the  all  or  nothing  funding 
policy: 

Let  Xj  =  1  if  the  modification  “i”  is  funded  and  0  if  not  funded 

Let  Vj=The  value  change  (benefit)  in  the  F-16  capabilities  due  to  implementation 

of  the  modification  i 

Let  Cj=The  total  cost  of  the  modification  i(total  number  of  aircraft  modified* 
acquisition  cost  for  an  engine) 

Let  B=Available  budget  for  a  portfolio  of  modifications 
Let  i =1..10  (The  alternatives  or  modifications  for  the  analysis) 

Maximize  the  total  value  =  ^Vj*Xi 

I 

Subject  to 

£Ci*Xi<=B 

i 

where 

Xj={0,l}  for  all  i 

The  optimal  portfolio  for  fleet  size  was  found  by  using  an  all  or  nothing  funding  policy 
since  this  research  is  concerned  about  the  overall  fleet  budget.  The  values  for  the  benefit 
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(Vj)  and  the  cost  (Q)  are  taken  from  the  Table  4.1 1.  The  process  for  mission  type  and 
war  capabilities  and  PW  engines  are  the  same.  The  only  change  is  the  figures  for  V)  due 
to  the  changing  benefits  of  the  modifications  for  different  scenarios  (mission  type  and 
war  capability  benefits  and  PW  engine  benefits)  as  explained  and  shown  previously.  The 
Excel  Solver  software  was  used  for  this  part  of  the  study.  The  modifications  are  coded 
with  capital  letters  for  the  efficiency.  Table  4. 14  shows  the  coded  modifications. 


Table  4.14  Coded  Modifications 


Engine  Types 

Alternatives 

Blocks  1 

Benefits 

Coded 

GE  100 

DEC  Upgrade 

30-40 

0.08913 

A 

GE  100 

Ejector  Nozzle 

30-40 

0.07061 

B 

GE  129 

Laser  Shock  Peen 

50 

0.04266 

C 

GE  129 

Turbine  Frame  Outer  Liner 

50 

0.02418 

D 

PW  229 

2nd  Fan  Stator 

52 

0.00708 

R 

PW  229 

#4  Bearing  Seals 

52 

0.00701 

F 

PW  229 

Oil  Filter  Housing 

52 

0.00700 

G 

PW  229 

2nd  Turbine  Blade 

52 

0.00420 

H 

PW  229 

Fuel  Nozzle  Damping 

52 

0.00312 

1 

PW  229 

Rear  Fan  Duct  Patch 

52 

0.00061 

J 

Table  4.15  shows  the  results  from  the  F-16  Capability  Enhancement  Model  with 
changing  budgets.  The  objective  function  (OBJ.  FUNC.)  value  is  the  total  benefit  of  the 
selected  portfolio  given  the  budget  constraint.  Right  hand  side  (RHS)  value  is  the  budget 
constraint  in  thousands  dollars.  The  RHS  is  increased  to  reflect  modification  sets 
produced  for  different  budget  constraints.  The  USED  column  is  the  money  in  thousands 
of  dollars  spent  for  the  specific  portfolio.  As  indicated,  the  total  cost  of  each 
modification  set  does  not  have  to  equal  the  budget.  This  represents  slack  in  the  budget 
constraint. 
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Table  4.15  Optimization  for  Fleet  Size 


OBJ.  ABCDEFGHI  J 


FUNC. 

0.01773 

0 

0 

0 

0 

0 

1 

1 

0 

1 

1 

RHS 

$1,000.00 

USED 

$924.00 

0.04587 

0 

0 

0 

1 

1 

1 

1 

0 

0 

1 

$2,000.00 

$1,856.57 

0.04899 

0 

0 

0 

I 

1 

1 

1 

0 

1 

1 

$3,000.00 

$2,511.07 

0.08146 

0 

0 

1 

1 

0 

1 

1 

0 

0 

1 

$5,000.00 

$4,657.55 

0.09585 

0 

0 

1 

1 

1 

1 

1 

1 

1 

1 

$10,000.00 

$9,885.92 

0.16646 

0 

1 

1 

1 

1 

1 

1 

1 

1 

1 

$120,000.00  $113,625.90 

0.18497 

1 

0 

1 

i 

1 

1 

1 

1 

1 

1 

$150,000.00  $146,104.50 

0.25558 

1 

1 

1 

i 

1 

1 

1 

1 

1 

1 

$250,000.00  $249,844.50 

Table  4.15  demonstrates  the  integer  programming  and  effective  sets  produced.  Figure 
4.10  demonstrates  the  benefit  versus  costs  of  different  modification  sets. 


Figure  4.10  Effective  sets  comparisons 


As  Figure  4.10  shows,  the  first  5  sets  (based  on  the  changing  modifications  in  the  sets) 
get  substantial  benefit  due  to  the  higher  slope  of  the  figure.  It  is  possible  to  produce  a 
total  benefit  of  0.1  while  spending  less  than  10  million  dollars.  However,  after  this  point 
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the  slope  decreases  dramatically  due  to  total  high  total  costs  which  means  that  the  GE 
100  engine  modifications  (A&B)  require  an  extensive  amount  of  money  for  the  benefits 
they  provide. 

Integer  programming  techniques  may  be  used  successfully  for  budgeting  profiles 
including  other  constraints.  Time  and  capacity  constraints  may  be  added  to  the  model. 
Political  constraints  may  also  be  added  to  include  decision-maker’s  opinions  and  concern 
in  the  study.  An  example  would  be  implementing  a  GE  engine  modification  for  every  2 
PW  modifications.  However,  this  is  beyond  the  scope  of  the  study  and  is  not  explained  in 
detail  in  the  research. 

4.5  What-lf  Analysis 

One  question  during  the  discussion  of  the  weights  was  how  the  value  model 
would  react  to  the  changes  of  sub-objectives’  weights.  Sensitivity  analysis  is  performed 
to  see  the  model  stability  for  changing  weights  of  the  evaluation  measures.  What-if 
analysis  tries  to  answer  the  changing  concerns  of  USAF  for  the  F-16  aircraft.  The  F-16 
fighter  can  be  used  for  purposes  like  training  and  testing  in  the  future.  Thus,  combat 
capability  may  not  be  the  biggest  concern  of  the  decision-maker.  Safety  or  costs  may 
become  more  important  than  combat  capability  due  to  the  conditions  in  the  Air  Force. 

The  global  weights  of  safety  and  operations  cost  objectives  were  varied  to  see  the  ranking 
differences  for  the  modifications.  Table  4.16  shows  the  results  for  the  first  case.  Safety 
is  considered  as  the  most  important  concern  in  the  first  case  with  a  global  weight  of  0.5. 
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Table  4.16  What-If  Case  1 


Global  Weights  Safety  =  0.5 

Operational  Cost  =  0.2 
Combat  Capability=  0.3 


Engine  Type 

Alternative 

Value 

Hypothetical  Best 

0.704 

GE  129 

Laser  Shock  Peen 

0.420 

PW  229 

2nd  Fan  Stator 

0.309 

GE  100 

DEC  Upgade(30) 

0.302 

GE  100 

DEC  Upgade(40) 

0.302 

GE  129 

Frame  Outer  Liner 

0.274 

GE  100 

Ejector  Nozzle  (40) 

0.245 

PW  229 

#  Bearing  Seals 

0.176 

PW  229 

Oil  Filter  Housing 

0.175 

GE  100 

Ejector  Nozzle  (30) 

0.164 

PW  229 

Fuel  Nozzle  Dump 

0.138 

PW  229 

2nd  Turbine  Blade 

0.130 

PW  229 

Rear  Fun  Duct  Patch 

0.026 

Hypothetical  Worst 

0.018 

The  Laser  Shock  Peenfor  GE  129  is  the  closest  modification  to  the  hypothetical  best 

J 

case.  Safety  being  the  most  important  concern,  2  Fan  Stator  for  PW  229  and  DEC 
upgrade  (Block3 0/40)  for  GE  100  are  the  other  important  modifications  for  the  F-16 
aircraft. 

The  performance  of  the  modifications  in  the  sub-model  mainly  depends  on  the 
data.  To  see  the  ranking  changes  for  the  F-16  Capability  Enhancement  Model,  the 
decision  weights  must  be  incorporated  to  the  model.  Using  decision  weights  may  provide 
better  insights  for  the  decision-maker  depending  on  the  changing  preferences.  The  Fleet 
Size  scenario  is  used  as  an  example  to  see  the  overall  ranking  changes.  Table  4.17  shows 
the  results  for  the  overall  ranking. 
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Table  4.17  The  Ranking  for  Fleet  Size  (What-If  Case  1) 


Engine  Type 

Alternatives 

Block 

Benefit 

GE  100 

DEC  Upgrade 

30-40 

0.1274 

GE  100 

Ejector  Nozzle 

30-40 

0.0827 

GE  129 

Laser  Shock  Peen 

50 

0.0529 

GE  129 

Turbine  Frame  Outer  Liner 

50 

0.0345 

PW  229 

2nd  Fan  Stator 

52 

0.0117 

PW  229 

#4  Bearing  Seals 

52 

0.0067 

PW  229 

Oil  Filter  Housing 

52 

0.0067 

PW  229 

Fuel  Nozzle  Damping 

52 

0.0052 

PW  229 

2nd  Turbine  Blade 

52 

0.0049 

PW  229 

Rear  Fan  Duct  Patch 

52 

0.0010 

The  overall  ranking  for  the  fleet  size  does  not  change  when  the  global  weight  of 
the  safety  is  changed  to  0.5  (compare  the  Table  4.16  and  Table  4.5).  The  difference 
between  the  global  weights  of  the  safety  and  combat  capability  is  not  big  enough  to 
create  changes  in  the  rankings.  This  is  due  to  the  overall  decision  weights  defined  by  the 
decision-maker.  The  sub-model  ranking  changes  are  reversed  by  the  decision  weights 
(top  tier).  This  interaction  limits  the  impact  of  the  sub-model  global  weights. 

The  second  case  focuses  on  the  operations  cost  of  the  modifications.  The  global 
weight  of  the  operations  cost  was  changed  to  0.5  for  this  scenario.  The  global  weight  for 
the  safety  is  0.3.  Combat  capability  is  considered  the  least  important  objective  for  this 
case.  Table  4.18  demonstrates  the  results  for  sub-model. 
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Table  4.18  What-Ifcase2 


Global  Weights  Safety  =  0.3 

Operations  Cost  =  0.5 
Combat  Capability=  0.2 


Engine  Type 

Alternative 

Value 

Hypothetical  Best 

0.891 

GE  100 

Ejector  Nozzle(40) 

0.609 

GE  100 

Ejector  Nozzle(30) 

0.407 

GE  129 

Laser  Shock  Peen 

0.406 

GE  100 

DEC  Upgade(30) 

0.361 

GE  100 

DEC  Upgade(40) 

0.361 

PW  229 

2nd  Fan  Stator 

0.338 

GE  129 

Frame  Outer  Liner 

0.308 

PW  229 

Oil  Filter  Housing 

0.253 

PW  229 

#  Bearing  Seals 

0.252 

PW  229 

2nd  Turbine  Blade 

0.239 

PW  229 

Fuel  Nozzle  Dump 

0.224 

PW  229 

Rear  Fun  Duct  Patch 

0.158 

Hypothetical  Worst 

0.151 

The  best  alternative  for  the  cost  is  Ejector  Nozzle  for  GE  100(Block  30/40)  engines.  The 
Laser  Shock  Peeti  and  DEC  upgrades  for  block  30  and  40  were  other  critical 
modifications  in  the  sub-model. 

Incorporating  the  decision  weights  into  the  model  gives  us  a  better  comparison 
capability.  The  overall  model  ranking  (ranking  for  F-16  Capability  Enhancement  Model) 
is  shown  in  Table  4.19.  The  decision  weights  may  provide  better  insights  for  the 
decision-maker  depending  on  the  changing  preferences.  The  fleet  size  scenario  is  used 


as  an  example  to  see  the  overall  ranking  changes. 
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Table  4.19  Ranking  for  Fleet  Size  (What-If  Case  2) 


Engine  Type 

Alternatives 

Block 

Benefit 

GE  100 

Ejector  Nozzle 

30-40 

0.2055 

GE  100 

DEC  Upgrade 

30-40 

0.1523 

GE  129 

Laser  Shock  Peen 

50 

0.0512 

GE  129 

Turbine  Frame  Outer  Liner 

50 

0.0388 

PW  229 

2nd  Fan  Stator 

52 

0.0128 

PW  229 

Oil  Filter  Housing 

52 

0.0096 

PW  229 

#4  Bearing  Seals 

52 

0.0096 

PW  229 

2nd  Turbine  Blade 

52 

0.0091 

PW  229 

Fuel  Nozzle  Damping 

52 

0.0086 

PW  229 

Rear  Fan  Duct  Patch 

52 

0.0060 

Due  to  its  low  operations  costs,  GE 100  Ejector  Nozzle  ( Block  30/40)  is  the  most 
important  modification  in  the  list.  Other  GE  100/129  engine  modifications  follow  this 
modification.  However,  small  changes  in  the  global  weights  of  objectives  do  not 
severely  change  the  overall  rankings.  When  the  decision  weights  were  defined  as  the 
fleet  size  ratios  (normalization  explained  previously),  the  GE  100/129  engine 
modifications  are  the  winners  and  are  still  highly  ranked  in  this  scenario. 

The  final  subject  in  this  chapter  is  the  sensitivity  analysis  of  the  study.  Sensitivity 
analysis  helps  to  determine  how  sensitive  the  value  model  is  to  the  weights  defined  by  the 
decision-maker  in  the  sub-model.  The  decision  weights  are  used  in  this  part  of  the  study 
as  explained  previously. 
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4.6  Sensitivity  Analysis 


Sensitivity  analysis  is  done  by  changing  the  global  weights  of  the  evaluation 
measures.  These  weights  represent  the  relative  importance  that  is  attached  to  changes  in 
the  different  evaluation  measures.  Making  these  global  weight  changes  for  one 
evaluation  measure  result  in  changing  the  global  weights  for  other  evaluation  measures. 
For  example,  increasing  the  global  weight  of  UER  drivers  in  Figure  4.1 1,  below,  results 
in  decreasing  the  global  weights  of  the  other  evaluation  measures.  If  large  changes  in  the 
ranking  happen  due  to  the  small  weights  changes,  the  sub-model  is  considered  weight 
sensitive.  Otherwise,  the  sub-model  is  insensitive  and  allows  room  for  error  when 
determining  decision-maker  weights.  Figure  4.11  represents  the  sensitivity  analysis’ 
results  for  aircraft  availability  (UER  drivers).  However,  this  research  was  planned  as  a 
pilot  study,  thus  the  sensitivity  graphs  may  only  provide  information  about  the  sensitivity 
of  the  sub-model  to  changes  in  the  weights.  The  data  gaps  and  lack  of  a  final  decision¬ 
maker  represent  the  main  disadvantages  of  the  model.  Therefore,  these  graphs  are 
displayed  to  demonstrate  the  sensitivity  analysis  technique. 

The  vertical  line  in  Figure  4.1 1  shows  the  current  weight  setting.  Crossing 
alternative  lines  show  the  changes  in  the  rankings  based  on  the  new  weighting  settings. 
Thus,  the  rankings  are  sensitive  to  the  weight  changes  for  UER  Drivers  evaluation 
measure.  Appendix  C  shows  the  results  for  sensitivity  analysis  for  other  evaluation 
measures. 
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Percent  of  Weight  on  UER  Drivers  Measure 


129  LaserShockPeen 
Hypot.  Best 
229  OilFilterHousing 
229  #4  BearingSeals 
129  FrameOuterLiner 
229  2nd  Turb.Blade 
100  Dec  Upg.30 
100  Dec  Upg.40 
229  2ndFanStator 
229  FuelNozzleDamp 
229  R.FunDuctPatch 
100  Ejector  Nozzle30 
100  Ejector  Nozzle40 
Hypot.  Worst 


Preference  Set  =  F-16  Capability  Enhancement  Model 


Figure  4.11  Sensitivity  of  Weight  for  UER  Drivers 


The  main  point  of  the  sensitivity  analysis  is  to  demonstrate  the  significance  of  the 
weightings.  Model  users  should  remember  this  property  of  the  analysis.  The  decision¬ 
maker  and  the  experts  of  the  problem  should  define  the  weights  of  the  model  depending 
on  the  need  of  the  USAF  for  better  use  of  the  model.  The  F-16  organizations  using  the 
model  must  be  careful  weighting  the  hierarchy. 


4.7  Summary 

The  F-16  Capability  Enhancement  Model  was  applied  to  analyze  ten  different 
aircraft  modifications  for  different  purposes.  The  alternatives  were  ranked  by  the  benefit 
they  provide  for  the  aircraft  under  different  considerations.  The  acquisition  costs  were 
incorporated  into  the  model  to  show  the  benefit  per  engine.  The  benefit/cost  ratio  was 
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applied  to  explain  the  benefit  per  dollar  invested.  The  benefit/cost  ratio  greedy  algorithm 
was  used  to  provide  better  insights  about  the  modifications’  total  acquisition  costs. 
Integer  programming  was  used  to  find  an  optimal  portfolio  for  the  F-16  aircraft. 

Different  effective  sets  were  introduced  and  total  costs  analyses  were  done. 

What-if  and  sensitivity  analysis  helped  to  understand  the  stability  of  the  model  to 
the  changes  in  the  weights.  Two  different  scenarios  were  used  in  the  what-if  analysis.  In 
the  first  case,  safety  was  accepted  as  the  most  important  concern.  In  the  second  case, 
operations  costs  were  defined  as  the  most  critical  issue.  The  sub-model  was  found  to  be 
weight  sensitive.  However,  when  the  decision  weights  were  incorporated  to  the  model, 
the  rankings  did  not  change  significantly.  Sensitivity  analysis  showed  that  the  sub-model 
was  weight  sensitive  and  highlighted  the  need  for  careful  weighting. 

Chapter  5  explains  the  overall  conclusions  and  includes  advice  for  future 
researches. 
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Chapter  5:  Conclusions 


5.1  Introduction 

The  objective  of  this  study  was  to  select  the  best  engine  modifications  for  F-16 
capability  enhancement.  This  single  objective,  which  is  defined  easily,  is  not  a  Simple 
target  to  achieve.  This  single  objective  is  composed  of  multiple  attributes.  The  research 
showed  that  extensive  effort  was  needed,  requiring  a  detailed  literature  review, 
development  of  a  complex  model,  and  analysis  to  provide  the  answer. 

Value  focused  thinking  was  demonstrated  as  a  potential  tool  for  solving  this 
multi  attribute  decision  problem.  This  thesis  effort  provides  a  way  of  organizing  and 
understanding  the  data  to  make  decisions  for  aircraft  modification  selection.  The 
following  section  explains  the  big  picture  of  the  entire  process. 

Research  and  development  organizations  have  to  rationalize  their  decisions 
depending  on  their  values.  Chapter  lexplained  this  issue  and  explained  the  difficulty  of 
this  problem  due  to  multiple  aspects.  An  organized  data  gathering  process  and  modeling 
approach  is  needed  to  help  the  decision-maker. 

Chapter  2  began  with  a  detailed  discussion  about  safety,  operational  cost,  and 
combat  capability  for  F-16  aircraft.  The  importance  of  decision  analysis  was  also 
explained.  Technology  selection  models  were  examined  after  the  discussion  about 
decision  analysis.  VFT  and  AHP  techniques  were  studied.  VFT  was  selected  due  to  its 
theoretical  foundations  and  its  unique  ability  to  allow  PrOACT  thinking.  Examples  of 
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VFT  were  introduced  to  show  its  capabilities.  The  chapter  was  completed  with  the 
characteristics  and  limitations  of  the  past  technology  selection  models. 

The  VFT  methodology  was  explained  in  Chapter  3.  The  overall  framework  was 
identified.  The  model  building  process  was  examined  to  give  better  insights  about  the 
development  process.  A  modified  VFT  approach  was  elucidated  to  improve  the 
flexibility  of  the  model.  Next,  the  values  of  the  F-16  organizations  were  clarified.  Single 
dimensional  value  functions  and  weights  were  solicited  to  complete  the  chapter. 

In  Chapter  4,  ten  modifications  were  analyzed  in  three  different  scenarios 
identified  by  the  decision-maker.  The  goal  was  to  determine  which  modifications 
provide  the  most  benefit  to  the  F-16  aircraft.  Cost  of  modifications  was  introduced  as 
another  aspect  the  problem.  Optimization  methods  were  used  to  minimize  the  impacts  of 
budget  constraints.  Finally  sensitivity  analysis  showed  that  the  model  was  sensitive  to 
the  changes  in  the  weights.  The  decision-maker  was  provided  the  needed  insights  from 
the  model  to  aid  in  the  modification  selection  process. 

5.2  Conclusions 

The  dimensions  of  the  decision  problem  (specific  to  the  F-16)  were  defined  as  the 
anticipated  values  of  the  ACC.  The  importance  of  an  organized  data  gathering  process 
was  highlighted  during  the  study  by  F-16  SPO  and  other  organizations  involved  in  the 
process.  A  mathematical  model  was  developed  by  using  the  anticipated  values  of  the 
ACC.  This  model  provided  a  systematic  approach  to  decision-making  process  and 
improved  their  current  techniques.  The  model  was  used  to  identify  the  benefits  of  the 
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modifications.  The  costs  for  modification  were  involved  in  the  process  to  demonstrate 
the  possibility  of  changing  benefits  per  dollar  invested.  Optimization  techniques  were 
used  to  show  that  selecting  a  portfolio  of  best  modifications  was  possible  for  F-16 
aircraft.  The  simplified  case  study  proved  that  the  multiattribute  decision-making 
techniques  could  be  applied  to  ASC  and  other  organizations’  decisions.  VFT 
successfully  analyzed  ten  engine  modifications.  The  model  built  in  this  study  can  save 
time  and  valuable  dollars  of  the  F-16  organizations.  The  most  important  impact  of  this 
research  is  providing  an  objective,  scientific  decision  support  system  for  the  ASC. 

GE  100/129  engine  modifications  are  the  most  beneficiary  modifications  for  the 
F-16  aircraft.  However,  they  score  poorly  when  cost  is  a  big  driver  for  the  decision.  The 
costs  of  the  GE  100/129  engine  modifications  must  be  decreased  for  better  results.  PW 
engine  modifications  have  mainly  low  acquisition  costs.  Due  to  their  low  acquisition 
costs,  they  are  more  applicable  for  low  cost  budgeting  profiles. 

ASC  should  implement  this  method  for  selecting  different  modifications.  The 
model  should  be  expanded,  validated,  and  verified  by  the  model  users.  This  research 
provides  structure  and  guidance  to  implement  this  decision  analysis  approach. 

5.3  Recommendations  for  future  research 

The  data  gathering  process  is  one  of  most  important  parts  of  the  problem. 
The  consequences  of  valid  data  gathering  on  the  study  must  be  understood  by  the 
organizations.  The  time  and  effort  required  to  gather  the  relevant  data  are  extensive. 
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Time  was  a  limiting  aspect  of  this  research.  The  F-16  aircraft  data  must  be  improved  to 
relax  some  of  the  data  assumptions  explained  in  Chapter  3. 

The  assumption  of  modification  independence  should  be  reviewed.  However, 
emphasis  should  be  placed  on  the  data  gathering  process  as  it  is  time  consuming  and 
complex.  Thus,  relaxing  many  assumptions  will  depend  on  the  data  improvements  versus 
the  analysis.  The  F-16  organization  or  users  of  the  model  must  recognize  these 
limitations  and  expend  time,  money,  and  man-hours  to  solve  these  problems. 

This  process  provides  a  new  approach  to  select  a  portfolio  of  modification.  It  is  a 
top-down  approach  to  show  that  VFT  can  be  applied  to  F-16  decision-making  processes. 
The  model  represents  the  general  aspects  of  the  problem.  Future  research  can  break  the 
problem  into  small  sections  and  construct  more  robust,  separate  models.  Individual 
hierarchies  will  catch  more  detail  and  address  issues  that  are  not  involved  the  original 
model. 

Other  multiattribute  decision  techniques  can  be  applied  to  this  problem  for 
comparisons  with  VFT.  These  comparisons  may  provide  additional  insight  to  build  an 
overall  model  and  analyze  results.  Even  AHP,  with  known  limitations,  may  be  applied  to 
problem  and  results  may  provide  different  insights  for  the  decision-maker.  The  decision 
weights  must  be  defined  clearly  since  they  change  the  rankings  of  the  alternatives  due  to 
various  preferences  of  the  decision-maker.  The  decision  weights  of  the  model  can  be 
determined  using  another  multiattribute  decision-making  techniques  to  help  the  decision¬ 
maker. 

The  involvement  of  the  decision-makers  and  experts  from  different  organizations 
is  vital  in  this  process.  The  key  elements  (values)  of  the  F-16  Capability  Enhancement 
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model  should  be  examined  by  the  high-level  decision-makers  and  experts  from  different 
F-16  organizations.  It  is  possible  that  some  of  the  means  objectives  can  be  replaced  with 
the  ends  objectives. 

The  model  used  in  the  study  is  designed  for  the  engine  modifications  only.  Future 
research  should  broaden  the  scope  of  the  study  to  the  all  modifications  that  enhance  the 
combat  capability  of  the  F-16  aircraft.  It  is  clear  that  the  hierarchy  will  change, 
especially  at  the  lower  levels  (evaluation  measures)  if  the  alternative  set  includes 
different  modifications. 

This  effort  should  be  expanded  to  the  other  fighter  aircraft  after  the  validation  and 
verification  process.  The  goal  is  to  build  an  overall  model  for  ACC  to  select  alternatives 
that  enhance  the  capabilities  of  the  war  fighter  in  a  proven  scientific  fashion.  However, 
this  is  only  possible  if  representatives  from  each  organization  are  involved  in  the  process 
and  discuss  their  values  and  the  overall  values  of  ACC.  A  final  model  could  be  built  to 
support  the  decision-making  process  of  ACC. 

Finally,  the  areas  for  uncertainty  in  the  model  must  be  identified  for  future 
studies.  The  experts  and  decision-makers’  opinions  about  the  uncertainties  should  be 
implemented  in  the  model.  Uncertainties  may  cause  ranking  changes  among  the 
modifications  depending  on  their  probabilities. 


5.4  Lessons  Learned 

This  section  explains  the  key  elements  in  the  decision-making  process.  The  first 
step  in  the  process  is  building  a  team  for  the  problem.  The  team  or  analyst  (student) 
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should  pick  an  overall  lead  and  a  lead  per  organization.  It  helps  in  assigning  the 
responsibilities.  Communications  between  student  and  leaders  should  begin  before  the 
first  meeting  to  find  the  support  needed  for  the  research.  The  student  must  identify  the 
key  decision-maker  in  the  process  prior  to  first  meeting.  The  student  has  to  keep  in  mind 
that  solving  the  multiattribute  decision-making  problem  is  a  team  effort.  Thus,  an  agenda 
must  be  set  including  schedule,  topics,  desired  attendees,  and,  expected  results  to  make 
sure  every  team  member  can  adequately  prepare.  Their  area  of  expertise  and  background 
might  be  valuable  to  fill  the  communication  gaps  among  team  members. 

The  first  meeting  is  the  most  important  meeting  in  many  aspects.  All  key  players 
in  the  problem  must  be  involved  in  this  meeting.  The  problem  must  be  defined  clearly. 
The  scope  of  the  study  must  be  identified.  Support  in  terms  of  time  and  money  for  the 
study  must  be  described  by  the  team.  The  expected  results  must  be  reviewed  to  set  a 
clearly  defined  goal.  The  analyst  has  to  understand  the  deliverables  of  the  decision¬ 
maker  in  the  first  meeting.  The  time  and  contents  of  the  deliverables  might  be  modified 
later  in  the  project. 

Scheduling  is  a  very  important  issue  for  stakeholders  in  the  project.  The  student 
must  keep  in  mind  that  clients  are  worried  about  solutions  not  graduation.  The  student 
has  to  set  the  schedule  due  to  his  or  her  needs,  like  workload  of  classes,  ability,  and  etc. 
The  timing  and  contents  of  the  schedule  must  be  modified  due  to  the  changing  needs  of 
the  stakeholders.  The  documentation  of  the  schedule  and  meetings  help  the  team  to  keep 
track  of  the  study. 

The  analyst  must  keep  in  mind  that  the  advisor  is  one  of  the  most  critical  players 
in  the  game.  The  analyst  has  to  keep  the  advisor  up-to-date.  Complex  problems  may 
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arise  during  the  process  and  the  experience  of  the  advisor  may  be  helpful.  If  needed, 
advisors  may  be  involved  in  the  process  as  a  part  of  the  team.  After  all,  the  decision¬ 
making  process  may  benefit  from  additional  experiences  in  the  academics  of  multi 
criteria  decision-making. 

The  following  equation  represents  a  good  decision-making  process: 

Effective  Decision=Quality  Thinking  *  Acceptance 

(Dr.  Deckro:  Synergistic  decision  making  brief) 


5.5  Summary 

This  chapter  explains  the  framework  used  for  this  research.  The  conclusions  are 
abridged  as  the  result  of  analysis  done  for  the  study.  The  possible  areas  for  improvement 
were  identified  to  help  the  future  researches.  The  process  steps  were  reviewed  to  help 
future  analyst.  The  key  elements  were  identified  to  complete  the  chapter. 
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APPENDIX  A:  F-16  Engines 


Engine  Model:  PW  -F-100  Family 

The  F100  family  of  engines  powers  the  FI5A-E  models  and  the  F16  A-D  models.  The  F100  is 
the  only  engine  to  power  the  F15  in  operational  service  and  has  the  fantastic  record  of  100.5 
aircraft  shot  down  without  ever  losing  an  FI  5  in  combat.  The  FI  6  is  the  front  line  air  to  ground 
aircraft  for  the  USAF.  The  F100  continues  to  be  the  safest  fighter  engine  in  history  with 
excellent  reliability.  The  F100  family  of  fighter  engines  evolved  from  Pratt  &  Whitney's  tradition 
of  dependable  engines.  The  first  F100  entered  service  in  1974.  More  than  6,400  FlOOs,  in  the  air 
forces  of  the  United  States  and  17  other  nations,  have  accumulated  more  than  14  million  engine 
flight  hours.  The  F100-PW-220  is  the  successor  to  the  F100-PW-100  that  powers  the  twin-engine 
Boeing  F-15  and  single-engine  Lockheed  Martin  F-16  fighters.  An  increased  performance 
engine,  the  F100-PW-229,  joined  the  U.S.  Air  Force  fleet  in  1991.  It  provides  22  percent  greater 
takeoff  thrust  for  the  Air  Force's  F-15E  dual  role  fighter  and  for  new  F-16s  (C/D  models).  The 
improved  version  can  be  installed  in  all  previous  F-16  models.  Pratt  &  Whitney  is  developing  an 
upgraded  version  of  the  PW-229,  called  the  F100-PW-232,  which  features  a  larger  and  more 
efficient  fan. 

PW  Military  Engines  on  line  available  at 
http://www.pratt-whitnev.com 
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Engine  Model:  F110-GE-100  and  F110-GE-400 


Description:  The  FI  10-GE-100  was  developed  to  provide  greatly  increased  performance  for 
later  models  of  the  F-16.  The  F110-GE-100  has  demonstrated  unrestricted  throttle  movement  and 
stall-free  operation  in  service,  and  has  set  the  benchmark  for  single-engine  fighter  reliability  and 
safety. 

The  F110-GE-400  was  selected  by  the  US  Navy  to  re-engine  their  existing  F-14A  fleet 
(transforming  them  into  F-14Bs)  and  to  be  the  production  standard  engine  for  new  F-14D 
aircraft.  The  -400  powered  F-14s  have  increased  range,  endurance, maneuverability,  and  can 
make  carrier  takeoffs  without  the  use  of  afterburning. 

Engine  Model:  F-110-GE-129 

Description:  The  F110-GE-129  is  a  derivative  of  the  proven  F110-GE-100  providing  increased 
performance,  enhanced  durability  and  even  greater  reliability.  New  features  include  a  full 
authority  digital  electronic  control  and  advanced  turbine  materials.  Capitalizing  on  the  success  of 
the  F110-GE-100,  the  -129  retains  84%  of  that  engine’s  basic  hardware  and  engine  architecture, 
which  were  designed  with  significant  future  growth  capability.  Since  entering  service  in  1992, 
the  FI  10-GE-129  has  proven  to  be  the  most  successful  fighter  engine  in  USAF  history. 

Because  of  the  FI  10  engine’s  proven  safety  track  record  and  heritage  of  high  reliability,  the 
F110-GE-129  has  been  chosen  to  power  more  than  75%  of  the  USAF’s  single  engine  F-16  Block 
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50/52*  aircraft.  Having  accumulated  more  than  400,000  flight  hours  in  the  Block  50,  the  FI  10- 
GE-129  has  established  a  track  record  of  excellence  that  is  unmatched  by  any  engine  in  its  class. 
Due  to  its  inherent  design  and  GE’s  unyielding  emphasis  on  six  sigma  quality,  the  engine  has 
demonstrated  unmatched  levels  of  on-wing  performance  retention.  No  F110-GE-129  has  ever 
been  removed  from  service  due  to  insufficient  performance!  The  F110-GE-129  has  also  been  the 
choice  of  the  Turkish,  Greek,  and  Japanese  air  forces  to  power  their  single  engine  F-16s.  434 
production  units  have  been  shipped  to  date.The  -129  is  also  qualified  on  the  F-15E  Strike  Eagle 
and  recently  completed  a  highly  successful  field  service  evaluation  on  this  application. 

GE  Military  Engines  online  available  at 
http://www.geae.com/militarv 
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APPENDIX  B:  Metamorphoses  of  the  model 


This  appendix  explains  the  different  models  and  their  problems  created  to  select 
the  best  engine  modifications.  Our  team  agreed  that  Combat  Capability,  Safety  and 
Operational  Cost  were  the  critical  objectives  of  the  model.  The  first  model  is  shown  in 
Figure  1. 
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Figure  1  The  First  Value  Model 


Some  problems  and  concerns  emerged  after  testing  the  model’s  robustness  for  the 
different  aspects  of  ACC’s  decision-making  process.  The  main  problems  were: 

•  How  to  deal  with  the  number  of  aircraft  effected  by  a  specific  modification. 

•  How  to  deal  with  the  special  missions  flown  by  specific  block  of  F-16. 
Normalization  by  aircraft  number  at  the  end  of  the  model  was  one  solution 

considered  for  solving  the  first  problem.  The  solution  to  the  second  problem  was  thought 


124 


to  have  been  already  addressed  in  the  model  by  the  metric  for  mission  type.  The  example 
shown  in  Table  1  help  us  understand  the  problem  better. 


Table  1  Example  for  the  first  model  problem 


Modification 

Type 

Type  of  A/C 
affected 

Normalization  Value 

The  value  for 
fundamental 
objective 

Final  Value  (The 
value  after 
normalization) 

A 

Block  30 

600/1400=0.43 
(Number  of  Block 

30  divided  by  Total 
A/C  number 

0.4 

0.4*0.43  =  0.17 

B 

Block  50 

50/1400=0.035 
(Number  of  Block 

50  divided  by  Total 
A/C  number 

0.8 

0.8*0.035  =  0.03 

i 

Modification  A  was  considered  better  than  B,  because  it  affected  more  aircraft. 
However,  the  decision-maker  said  that  this  assumption  was  not  completely  true  since 
ACC  and  F-16  SPO  would  be  willing  to  select  modification  B  to  modify  Block  50 
aircraft.  Block  50  aircraft  fly  a  very  critical  mission  called  suppression  of  enemy  air 
defense  (SEAD). 

The  first  solution  to  the  problem  was  basically  changing  the  value  model  so  that 
we  could  give  more  value  to  the  mission  type.  The  second  model,  as  shown  in  Figure  2, 
was  developed  to  capture  the  decision-  maker’s  opinions. 
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Figure  2  A  Solution  to  the  Mission  Type  Problem 


Mission  type  in  this  model  had  a  global  weight  of  0.48  (0. 6*0.8).  The  metric  for 
Mission  type  was  planned  to  be  a  discrete  function.  It  was  a  simple  yes/  no  question  for 
the  calculations.  Thus,  if  any  modification  affected  the  SEAD,  it  would  get  a  total  value 
of  0.48  for  combat  capability  and  possibly  reach  a  score  of  1  for  total.  If  a  modification 
was  not  effecting  SEAD,  it  would  get  0  for  combat  capability  and  possibly  reach  a  total 
score  of  0.52  in  a  best  case.  However,  normalization  by  aircraft  would  give  a  lower  score 
for  Block  50  (only  the  blocks  that  can  fly  the  SEAD)  since  they  had  a  very  small  fleet 
size  compared  to  other  blocks  no  matter  how  we  the  model  was  changed.  Therefore,  the 
model  was  not  approved  as  a  solution  to  the  problem,  and  a  better  and  more  flexible 
solution  was  needed. 
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Further  studies  and  examination  of  the  model  created  the  second  solution.  New 


metrics  were  added  to  the  model.  A  higher-level  tier  was  connected  to  the  model  to  deal 
with  mission  and  aircraft  numbers.  The  second  solution,  (a  modified  VFT  approach)  is 
shown  in  Figure  3. 


Figure  3  Final  Model 
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APPENDIX  C:  Sensitivity  Analysis  Graphs 

The  sensitivity  analysis  graphs  show  the  model  sensitivity  to  the  changes  in  the  weights. 
The  graphs,  shown  below,  demonstrate  that  our  sub-model  is  weight  sensitive. 
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The  graphs  for  MMH  and  O&S  Parts  are  weight  sensitive  to  the  decreases  in  the  overall 
weights.  On  the  other  hand,  WAPM  is  weight  sensitive  to  the  increases  in  the  weights. 
Number  of  Class  A’s(I-T/F),  and  Number  of  Class  A’s(M-T/F)  evaluation  measures  are 
sensitive  to  both  increases  and  decreases.  O&S  Labor  is  the  only  evaluation  measure  that 
can  have  room  for  errors.  The  model  users  should  remember  this  property  of  the 
analysis.  The  decision-maker  and  the  experts  of  the  problem  should  define  the  weights  of 
the  model  depending  on  the  need  of  the  USAF  for  better  use  of  the  model.  The  F-16 
organizations  using  the  model  have  to  be  careful  about  the  weighting  process. 
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APPENDIX  D:  Modifications’  Data 
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ECP  Data  Current  Proposed 

Hardware  Cost  $1,101  $2,513 

Depot  Replacement  Interval  (hrs)  3000  3000 


Depot  Scrap  Rate  100%  100% 

Depot  Labor  to  Remove  &  Replace  (Hrs)  44  44 

Lifetime  Depot  Hardware  Replacements  150  150 

Lifetime  Depot  Labor  Hours  6578  6578 
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Non-RTOC  Data 


Lifetime  Field  Maintenance  Avoided  (MMH)  175 

Lifetime  Field  Inspection  Hours  Avoided  (MMH)  2,243 

Total  Maintenance  Hours  Avoided  _ 2,418 

MMH  / 1 000  EFH  Avoided  due  to  modification  5.39 


0 

0 

0 

> 


OO  CD  ^  CM 

dodo 


CD  00  CD 
LU  S  S 

o  d 


co  in  co  r- 

y-  rr  iO  O 

in  oo  i-  m 
cm"  co"  o"  oo" 

CD  00  CD 


69-  60  CD  CD 


0 

0 

LL 


CO 

a> 

c 

*CD 

c 

LLI 

=fc 


fv'*  r*-*  cd 
00  CO  CM 


Tt  N. 

CM  CM  t-  ° 


CM 

CM 

00 

o 

O 

o 

T — 

“O 

0 

£  0 

LO 

CD 

CD  CD 

0  2 
*  g  s 

s  i  s 

O  c  0 
1-  <  Q- 

o- 

32 

o 

_c 

CO 

0 


H 

0 

2  0 
<0  CD 

*  5 

>  o 

c3  0 
*=  CD 
0 

CD  E 
0 
JQ 
O 


0 

o 

■a 

o 


co 

0 

o 

a 


"O 

0 

0 

S  Q_ 

2  o 

0  n* 

CD  u~ 

'^Z  i 

0) 

-1  0 
0  D 

•M 

£  2 

a>  as 

-4— » 

0  0 
DC  DC 

<  < 
0  0 
0  0 
0  0 

o  o 

0  0 

a 

0  0 

a  a 


o 

.Q 

0 

C 

o 

0 

0  0 

<5  1 


0 
O 

O 

CC  0  £  £ 


1  |§ 
0  -o  O 
0  £ 
0  o 
X3  t- 
D  0 
O  CC 

—  0 

© 

0  .9 

o  c 

LU 

S  I 

II 


DC  o 

0  O 

0  ^ 
C  U— 

•03  2 

LU 

0  _ 
JS  0 


0 

DC 


0 

D 

T5 

0 

O 

CD 

c 

o 


r-  C\)  0  Tf  LO 


O 

h- 


0 

"0 

0 

c 


0  0  0 
0  0  0 
>  >-  > 


0 

0 

> 


0 

O 


132 


CM 

<o 


mcMcoa>moLointn 

COt-CMCMCMOOOOO 


T3 

0 

C/j  CD  CD 
o  ^  CD 

a  °  ° 

odd 


TJ 

0) 

•*>* 

CO 

CD 

'M* 

TJ 

0 

</> 

o 

O 

o 

CD 

O 

O 

0s 

o 

C\J 

LO 

CD 

o> 

T— 

't~ 

a 

o> 

CD 

O 

LO 

CM 

i 

d 

d 

o 

v. 

CL 

09- 

00 

CM 

.2 

00 

^r 

c 

r^ 

m 

o 

h- 

CD 

0 

in 

o 

00 

CD 

k. 

o 

CD 

c 

d 

d 

3 

CM 

o 

69- 

2  w 

0  E 

s 

O  0 

CO 

CO 

0  O 

to 

L_ 

-C 

CL  -2 

Q 

CD  Q. 

X 

Cl 

=  £  2? 

LL 

O 

£ 

LU 

LLi 

0 

©  £  o 

C 

>  ro  X 

r\  > 

X  0  > 
0)  o 
.9  o  cj 


LL 

<L 

0 

05 

+3 

CO 

O 

V— ' 

o 

o 

0 

> 

0 

>* 

-C 

-C 

C 

*a 

CO 

CO 

*D) 

o 

0 

0 

2 

D 

0 

1— 

0 

c 

JC 

o 

O 

-C 

-C 

UJ 

1- 

00 

H 

X 

1- 

133 


Field  Scheduled  Event  Rate  (Events  /  1 000  EFH)  0.2389  0.0000 

Total  Field  Event  Rate  (per  1000  EFH)  0.2434  0.0005 

Field  Scrap  Rate  100%  100% 


Field  Labor  to  Remove  &  Replace  (Hrs)  108.9  1Q8.9 

Remove  Engine  19  19 

Remove  &  Replace  Hardware  83.4  83.4 

Engine  Test  6.5  6.5 

Lifetime  Field  Hardware  Replacements  109.14  0.22 
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Does  Mod  Get  Problem  Below  Safety  Threshold?  YES 

Delta  Class  A  Rate  (Initial  -  Mitigated)  0.552 

Delta  Class  A  Rate  (Mitigated  -  Proposed)  0.105 


Retrofit  Cost 

Unit  Kit  Cost  (includes  depot  installation  labor)  $  1 9,600 

Install  Engines  Retrofit  Cost  $  1 ,274,000 

Spare  Engines  Retrofit  Cost  $  235,200 

Total  Retrofit  Cost  $  1,509,200 
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Table  3  PW  229  Fuel  Nozzle  Damping 
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Does  Mod  Get  Problem  Below  Safety  Threshold?  YES 

Delta  Class  A  Rate  (Initial  -  Mitigated)  0.000 

Delta  Class  A  Rate  (Mitigated  -  Proposed)  0.1 03 


Retrofit  Cost 

Unit  Kit  Cost  (includes  depot  installation  labor)  $  8,500 

Install  Engines  Retrofit  Cost  $  552,500 

Spare  Engines  Retrofit  Cost  $  1 02,000 

Total  Retrofit  Cost  $  654,500 
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Table  4  GE  100  DEC  Upgrade  (Block  30) 
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Non-RTOC  Data 

Lifetime  Field  Maintenance  Avoided  (MMH)  8,022 

Lifetime  Field  Inspection  Hours  Avoided  (MMH)  0 

Total  Maintenance  Hours  Avoided  8,022 
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Table  5  GE  100  DEC  Upgrade  (Block  40) 
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Non-RTOC  Data 

Lifetime  Field  Maintenance  Avoided  (MMH)  8,022 

Lifetime  Field  Inspection  Hours  Avoided  (MMH)  0 

Total  Maintenance  Hours  Avoided  8,022 


Safety  Data 

Does  Mod  Get  Problem  Below  Safety  Threshold?  YES 

Delta  Class  A  Rate  (Initial  -  Mitigated)  0.000 
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Field  Unscheduled  Event  Rate  (Events  / 1 000  EFH)  0.331  0 

Field  Scheduled  Event  Rate  (Events  /  1000  EFH)  0.0000  0.0000 

Total  Field  Event  Rate  (per  1000  EFH)  0.3310  0.0000 

Field  Scrap  Rate  0.30%  0% 


Field  Labor  to  Remove  &  Replace  (Hrs)  32.5  32.5 

Remove  Engine  19  19 

Remove  &  Replace  Hardware  12  12 

Engine  Test  1.5  1.5 

Lifetime  Field  Hardware  Replacements  0.45  0.00 
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Does  Mod  Get  Problem  Below  Safety  Threshold?  YES 

Delta  Class  A  Rate  (Initial  -  Mitigated)  0.000 

Delta  Class  A  Rate  (Mitigated  -  Proposed)  0.002 


Retrofit  Cost 

Unit  Kit  Cost  (includes  depot  installation  labor)  $  354 

Install  Engines  Retrofit  Cost  $  23,01 0 

Spare  Engines  Retrofit  Cost  $  4,248 

Total  Retrofit  Cost  $  27,258 
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Scheduled  %  at  O&l-Level  requiring  Repair  0%  0% 

Scheduled  Repair  Cost  at  O&l-Level  $0  $0 

Scheduled  %  Returned  to  Depot  0%  100% 

Scheduled  Man-Hours  at  Depot  0.0  0.0 


Scheduled  %  at  Depot  requiring  Repair  0%  0% 

Depot  Repair  Cost  $0  $0 

Scheduled  %  Scrapped  at  Depot  0%  100% 

Depot  Hardware  Scrap  Cost  $0  $239 

Lifetime  Scheduled  Hardware  Repair  Cost  $0  $0 
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RTOC  Data 

Lifetime  Hardware  Avoided  $  3,284 

Miscellaneous  Hardware  Cost  Avoidance  $ 


Lifetime  Depot  Labor  Avoided  J> _ ; 

Total  Lifetime  Cost  Avoidance  $ _ 3,284 

Hardware  $  / 1 000  EFH  _$ _ 7.32 

Labor  $/ 1000  EFH  $ 
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Year  3 
Year  4 
Year  5 
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Table  8  2na  Turbine  Blade 
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Scheduled  %  at  O&l-Level  requiring  Repair  72%  0% 

Scheduled  Repair  Cost  at  O&l-Level  $28,193  $39,648 

Scheduled  %  Returned  to  Depot  28%  1 00% 

Scheduled  Man-Hours  at  Depot  0.0  0.0 


Scheduled  %  at  Depot  requiring  Repair  0%  30% 

Depot  Repair  Cost  $14,400  $14,400 

Scheduled  %  Scrapped  at  Depot  100%  70% 

Depot  Hardware  Scrap  Cost  $28,193  $39,648 

Lifetime  Scheduled  Hardware  Repair  Cost  $6,354,650  $1 ,352,389 
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RTOC  Data 

Lifetime  Hardware  Avoided  $  7,277,901 

Miscellaneous  Hardware  Cost  Avoidance  $ 


Lifetime  Depot  Labor  Avoided  J> _ - 

Total  Lifetime  Cost  Avoidance  $ _ 7,277,901 

Hardware  $  /  1 000  EFH  _$ _ 16,227.20 

Labor$/ 1000  EFH  $ 
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Year  3  17  26% 

Year  4  0  0% 

Year  5 
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Table  9  GE  100  Ejector  Nozzle  (Block  30) 
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Non-RTOC  Data 

Lifetime  Field  Maintenance  Avoided  (MMH)  753,680 

Lifetime  Field  Inspection  Hours  Avoided  (MMH)  0 

Total  Maintenance  Hours  Avoided  753,680 


Safety  Data 

Does  Mod  Get  Problem  Below  Safety  Threshold?  YES 

Delta  Class  A  Rate  (Initial  -  Mitigated)  0.000 
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Total  weighted 

Annual  Percentage  0.762 


Table  10  GE  100  Ejector  Nozzle  (Block  40) 
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Non-RTOC  Data 

Lifetime  Field  Maintenance  Avoided  (MMH)  753,680 

Lifetime  Field  Inspection  Hours  Avoided  (MMH)  0 

Total  Maintenance  Hours  Avoided  753,680 


MMH  / 1000  EFH  Avoided  due  to  modification  575.47 
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Table  11  GE  129  Turbine  Frame  Outer  Liner 
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Scheduled  Repair  Cost  at  O&l-Level 
Scheduled  %  Returned  to  Depot 

Scheduled  Man-Hours  at  Depot 
Scheduled  %  at  Depot  requiring  Repair 
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Lifetime  Hardware  Avoided  $  1 ,626 

Miscellaneous  Hardware  Cost  Avoidance  $ 

Lifetime  Depot  Labor  Avoided  J> _ 38.415 


Total  Lifetime  Cost  Avoidance  $ _ 40,041 

Hardware  $  /  1 000  EFH  _$ _ 1.21 

Labor  $  / 1 000  EFH  $  28.57 
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Year  5 
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Scheduled  %  at  O&l-Level  requiring  Repair  0%  0% 

Scheduled  Repair  Cost  at  O&l-Level  $0  $0 

Scheduled  %  Returned  to  Depot  100%  100% 

Scheduled  Man-Hours  at  Depot  0.0  0.0 


Scheduled  %  at  Depot  requiring  Repair  0%  0% 

Depot  Repair  Cost  $0  $0 

Scheduled  %  Scrapped  at  Depot  10%  10% 

Depot  Hardware  Scrap  Cost  $28,661  $49,423 

Lifetime  Scheduled  Hardware  Repair  Cost  $0  $0 
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RTOC  Data 

Lifetime  Hardware  Avoided  $  (1 ,827,486) 

Miscellaneous  Hardware  Cost  Avoidance  $ 


Lifetime  Depot  Labor  Avoided  J> _ 792,837 

Total  Lifetime  Cost  Avoidance  $ _ (1 ,034,649 

Hardware  $  / 1 000  EFH  _$ _ (1,359.13; 

Labor  $/ 1 000  EFH  $  589.65 
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Year  3  72  29% 

Year  4  53  21% 

Year  5  0  0% 
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APPENDIX  E:  Macros  created  for  value  functions 


The  Visual  Basic  for  Application  functions,  shown  below,  converts  the  alternatives’  score 
to  values  using  single  dimensional  value  functions  based  on  the  preferences  of  the 
decision-maker  under  each  objective. 


Function  UER(score) 

If  score  >=  0  And  score  <=  0.07 14  Then-t- 
Select  Case  score 
Case  Is  >=  0  And  score  <=  0.07 14 
UER  =  (score  -  0)  /  (0.0714  -  0) 

End  Select 
Else 

MsgBox  "The  score  is  not  within  the  range  of  evalaution  measure" 
End  If 

End  Function 


Function  Delta  1  (score) 

If  score  >=  0  And  score  <=  0.7  Then 
Select  Case  score 
Case  Is  <=  0.091 

Deltal  =  (score  -  0)  /  ((0.091  -  0)  *  10) 

Case  Is  <=  0.7 

Deltal  =  (1  -  Exp(-(score  -  0.091)  /  0.5))  /  (1  -  Exp(-(0.7  -  0.091)  /  0.5)) 
End  Select 

Else 

MsgBox  "The  score  is  not  within  the  range  of  evalaution  measure" 

End  If 

End  Function 


Function  Delta2(score) 

If  score  >=  0  And  score  <=  0.7  Then 
Select  Case  score 
Case  Is  <=  0.091 

Delta2  =  (score  -  0)  /  ((0.091  -  0)  *  10) 

Case  Is  <=  0.7 

Delta2  =  (1  -  Exp(-(score  -  0.091)  /  0.2))  /  (1  -  Exp(-(0.7  -  0.091)  /  0.2)) 

End  Select 

Else 
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MsgBox  "The  score  is  not  within  the  range  of  evalaution  measure" 
End  If 

End  Function 


Function  Install(score) 

If  score  >=  0.4  And  score  <=  1  Then 
Install  =  (score  -  0.4)  /  (1  -  0.4) 

Else 

MsgBox  "The  score  is  not  within  the  range  of  evalaution  measure" 
End  If 

End  Function 


Function  Partscost(score) 

score  =  score  +  1359 

If  score  >=  0  And  score  <=  100000  Then 

score  =  score  /  100000 

Partscost  =  (1  -  Exp(-(score  -  0)  /  -0.5))  /  (1  -  Exp(-(1  -  0)  /  -0.5)) 
Else 

MsgBox  "The  score  is  not  within  the  range  of  evalaution  measure" 
End  If 

End  Function 


Function  Laborcosts(score) 

If  score  >=  0  And  score  <=  5000  Then 
score  =  score  /  5000 

Laborcosts  =  (1  - Exp(-(score -0)/-l))/(l  - Exp(-(1  - 0) / -1)) 

Else 

MsgBox  "The  score  is  not  within  the  range  of  evalaution  measure" 
End  If 

End  Function 


Function  MMH(score) 

If  score  >=  0  And  score  <=  600  Then 
score  =  score  /  600 

MMH  =  (1  -  Exp(-(score  -  0)  /  -1))  /  (1  -  Exp(-(1  -  0)  /  -1)) 

Else 

MsgBox  "The  score  is  not  within  the  range  of  evalaution  measure" 
End  If 

End  Function 
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Function  DWV  below,  incorporates  the  decision  weights  into  the  sub-model.  In  a 
spreadsheet,  it  compares  the  aircraft  types  (Bltypel  and  Bltype2)  and  if  they  are  same,  it 
multiplies  the  value  with  the  decision  weight  for  that  specific  block  to  find  the  value  of  a 
modification  for  a  specific  block  type. 


Function  DWV(Bltypel,  value,  Bltype2,  weight) 
If  Bltypel  =  Bltype2  Then 
DWV  =  value  *  weight 
Else 

DDWV  =  0 
End  If 

End  Function 
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